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PEEFACE 


This  report  is  the  fifth  •  annual  report  from ‘the  Data  Base  Group  in  the  Computer 
Systems  Research  Group.  It  iis  a  collection  of  pap>ers  of  members  in  the  group 
detailing  the  research  that  has  been  done  in  the  preceding  twelve  to  eighteen  months. 
As  with  the  last  report  (Omega  Alpha)  the  major  thrust  of  the  research  reported  here 
is  in  !the  larea  lof  office  information  systems.  -As  the  ibasis  of  our  office  information 
systems  neseairch  is  data  base  management,  one  will  Ifind  a  close  relationship  of  the 
work  reported  here  to  data  base  raan£gement.  The  major  difference  ’ivith  the  work  in 
data  base  management  is  in  the  increased  flexibility  and  functionality  found  in  all 
aspedts  of  office  information  sy.stems  research. 

The  first  area  of  office  mformation  ^sterns 'research  covered  in  this  report  is  that 
of  user  interfaces.  Office  mformation  systems  requireimuch  more  flexible  and  easy  to 
use  jquery  languages  taind  editors  than  are ‘traditionally  provided.  In  addition,  media 
other  than  merely  alkeyboard  need  Ito  be  investigated  and  integrated  into  the  user 
interface.  The  interactive  query  lamgutage  described  in  the  paper  by  Lochovsky  and 
Tsichritzis  was  prompted 'by  a  need*to  hafve  a  very  easy  to  use  language  for  home 
videotex  use;  ftiowever,  similar  ideas  are  directly  applicable  to  office  information 
systems  query Uanguages.  TThejpaper  by  Mendetzon  describes  the  design  of  a  data  base 
editor  modeled  after  the  UNIXIED  text  editx>r  that  allows  highly  interactive  access  to 
and  modification  of  a  data.  base.  Tinally,  the  paper  by  Lee  describes  a  voice  response 
system  that  allows  electronic  forms  to  ’’talk"  to. the  user  as  well  as  be  read  by  him. 

A  second  area  of  office  .inf crmati on  eystems  research  covered  here  deals  with  the 
modeling  of  the  officse  so  that  office  data  can  be  represented  by  a  computer.  The  paper 
by  Gibbs  land  that  by  Martin  and  Tsichritzis  deal  with  this  problem  in  two  slightly 
different  «ways.  The  iGibbs  paper  takes  a  semantic  modeling  approach  to  try  and 
capture  data  aspects  and  communication  aspects  in  an  office.  It  takes  its  major 
concepts  fromieirtificial  intelligence  and  semantic  data  models  and  tries  to  represent 
many  different  data  types  —  text,  graphics,  voice  —  in  the  same  model.  The  Martin 
and  Tsichritzis ipaper  takes  a  communication  approach  to  modeling  offices.  The  model 
described  is  derived  from  the  Smalltalk 'and  Actor  .approaches  that  use  message 
passing  and  pnocedunal  representation  within  the  formalism  to  communicate  among 
user  ^ents. 

A  third  research »area  covered  in  this  report  deals  with  analyzing  information  and 
data  flow  within  officjes.  In  the  presence  of  procedures  that  may  run  without  user 
intervention,  one  would  like  'to  be  able  to  determine  that  such  procedures  run 
correctly.  The  paper^by  Nierstrasz  and’ Tsichritzis  proposes  a  model  for  representing 
the  flow  df  data  within  an  ofiiGe  information  system  and  a  method  for  analyzing  the 
correctness  of  the  flow. 

A  final-. research  area  dealt  with  indhas  report  is  thal  of  storing  and  searching  large 
files  df  text.  One  would  like  to  be  able^to  search  text  for  arbitrary  words  or  phrases.  In 
a'file  'of  thousands  or  hundreds  of  thousands  of  text  messages  (e.g.,  letters)  scanning 
all  messages  sequentially  is  not  a  realistic  approach.  Traditional  indexing  schemes 
for  text  can  ret^uire  from  fifty  to  two  hundred  percent  additional  storage  for  the  index. 


In  the  paper  by  Tsichritzis  and  Chris todoulakis  and  that  by  Christodoulakis  and 
Faloutsos,  a  scheme  for  indexing  text  that  minimizes  storage  requirements  for  the 
index  and  yet  allows  efficient  searching  of  the  text  is  described. 

All  of  the  ideas  presented  in  the  papers  in  this  report  have  been  and  are  continuing 
to  be  developed  further.  In  addition,  they  are  being  incorporated  into  actual 
prototype  systems  being  developed  by  the  group  to  determine  their  efficacy  and 
usefulness. 


F.H.  Lochovsky 
Editor 
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ABSTRACT 


The  amount  of  information  that  is  available  in  computerized  form  is  growing 
steadily.  Two  developments  in  recent  years  —  the  growth  of  communication  networks 
and  the  advent  of  videotex  systems  —  provide  the  potential  for  edlowing  the  public 
convenient  access  to  this  information.  In  this  paper  we  explore  the  nature  of  the 
mechanism  by  which  people  in  their  homes  using  a  videotex  information  service  might 
access  data  bases  external  to  the  videotex  system  itself.  We  first  consider  the  process 
of  accessing  a  data  base  (querying)  in  terms  of  the  user  interaction  characteristics  of 
the  process.  We  divide  the  querying  process  into  tbj-ee  parts:  request,  reply  and 
dynsimics.  For  cdl  three  parts,  certain  characteristics  are  identified  that  describe  the 
parameters  of  the  user  interaction.  We  then  propose  some  requirements  that  a  query 
language  for  external  data  bases  should  have.  Wfith  these  requirements  in  mind  we 
detail  a  design  for  such  a  query  language. 


*  This  research  was  sponsored  by  the  Department  of  Communications,  Ottawa.  Canada 
under  Department  of  Supply  and  Services  contract  serial  no.  0SU80-00124. 

"^In  Proc.  Inti.  Conf.  on  Very  Large  Data  Bases,  Mexico  City, 
Mexico,  Sept.  1982. 
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1  INTRODUCTION 

Recent  years  have. seen  a  substantial  growth  in  the  number  of  on-line,  computer- 
based  information  sources  (data  bases).  Many  of  these  on-line  data  bases  provide 
information  to  the  public  (e.g.,  library  systems)  airline  systems  and  government 
information  services.  However,  the  public  generally  does  not  have  convenient  and 
direct  access  to  these  data  bases.  Instead,  some  intermediary  queries  the  data  base 
and  supplies  the  response,  or  the  information  is  available  in  a  non-selective  manner  in 
fixed  locations. 

Another  development  has  been  the  installation  and  growth  of  communication 
networks.  These  communication  networks  allow  data  bases  to  be  queried  remotely  at 
reasonable  cost.  This  in  turn  raises  the  possibiUty  of  permitting  access,  by  the  public, 
to  on-line  "consumer  information"  data  bases.  An  important  question  in  this  context  is 
from  where  and  how  is  this  access  to  be  permitted. 

A  third  development  has  been  the  emergence  of  videotex  systems.  Videotex 
systems  eire  interactive,  visual  communication  systems  intended,  in  part,  to  permit 
public  access  to  external  data  basesh  TeLidon  is  an  example  of  such  a  system  [Sown,  et 
2il.,  1978].  The  interaction  with  the  system  is  via  a  suitably  modified  television  receiver 
which  acts  as  the  terminal  display.  Input  is  accomplished  by  a  keypad  or  keyboard 
which  allows  certain  data  to  be  selected  for  display. 

Given  that  the  public  will  eventually  have  access  to  videotex  systems,  several  issues 
relating  to  user  interaction  with  these  systems  arise  [Sown  et  al.,  1979].  One  of  these 
issues  concerns  how  the  public  vnli  request  information  from  the  various  external  data 
bases  that  will  be  available  to  them.  Currently,  a  number  of  different  languages  each 
with  its  own  S3mtax  and  vocabulary  exist  for  querying  data  bases  [McDonald  and 
Stonebraker,  1975;  Denny,  1977;  Zloof,  1977;  Codd  et  al.,  1976,  Ellis  and  Nutt,  1980; 
Herot,  1980].  It  seems  intuitively  undesirable  for  a  user  of  a  videotex  system  to  have  to 
learn  and  remember  a  different  query  language  for  each  external  data  base  that  will  be 
accessed.  To  help  ensure  user  acceptance  of  videotex  information  retrieval  services, 
there  should  be  one  way  of  querying  externcd  data  bases  via  videotex  systems  that  is 
easy  to  learn  and  use  by  the  public.  Using  this  query  language,  people  can  access 
information  in  any  external  data  base  that  is  connected  to  the  communication 
network.  In  this  paper  we  establish  some  user  interaction  criteria  for  evaluating 
interactive  query  languages  and  propose  a  particular  approach  to  a  query  language  for 
external  data  bases. 


2  USER  INTERACTION  PARAMETERS 

The  general  area  of  person-computer  interaction  has  been  the  subject  of  much 
study  [Martin,  1973;  Gilb  and  Weinberg,  1977;  Guedj  et  al.,  1980;  Schnelderman,  1980; 
Mehlmann,  1981].  In  the  area  of  interactive  query  languages  the  studies  are  far  fewer 
[Reisner,  1981].  Applying  some  of  the  results  from  general  person-computer 
Interaction  and  the  studies  on  query  leinguages,  let  us  examine  the  process  of  querying 
a  data  base  from  the  user's  viewpoint.  The  querying  process  can  be  broken  up  into 


1.  By  an  external  data  base  we  mean  one  whose  contents  and  format  are  not  under  the  control  of  the 
information  delivery  (videotex)  system. 
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three  parts;  request,  reply  and  dynamics. 

For  each  part  of  the  querying  process  certain  characteristics  can  be  identified. 
These  characteristics  .describe  the  parameters  of  user  interaction.  The  "values"  that 
these  parameters  have  in  a  given  query  language  determine  whether  or  not  the  query 
language  has  "desirable"  characteristics  with  regard  to  the  person-computer  interface. 
In  the  rest  of  this  section  we  will  identify  and  define  several  cheracteristics  for  each 
part  of  the  querying  process.  For  a  more  detailed  discussion  of  these  parameters  see 
[Lochovsky  and  Tsichritzis,  1981]. 

2. 1  Request 

Request  deals  with  the  formulation  of  a  query  to  the  system  by  the  user.  The  user 
informs  the  system  of  some  action  that  he  wishes  it  to  do  for  him.  We  are  principally 
concerned  here  with  requests  for  stored  information.  Usually  some  form  of  "language" 
that  Is  mutually  acceptable  .to  the  user  and  the  system  Is  used  to  specify  a  request. 
This  language  can  take  several  forms  (e.g.,  visual,  verbad,  pantomime,  etc.).  Six 
characteristics  of  formulating  requests  are  important  for  eveduating  Interactive  query 
languages. 

1.  Keystrokes  are  a  quantification  of  the  amount  of  user-supplied  input  required 
before  the  system  fully  understands  the  user’s  request. 

2.  Commands  are  the  set  of  instructions  a  user  has  available  to  tell  the  system  what 
action  it  should  perform. 

3.  Formulation  of  a  request  concerns  how  difficult  it  is  to  specify  a  request  that  is 
incorrect  either  syntactically  or  semantically. 

4.  SeLectiuity  Is  the  ability  of  the  user  to  specify  ajs  precisely  as  possible  that  data 
which  he  wishes  to  retrieve. 

5.  Uniformity  concerns  the  independence  of  the  query  language  from  the  type  of  data 
base  it  is  accessing  and  the  application  for  which  the  language  is  being  used. 

6.  Customizing  of  a  query  language  deals  with  matching  the  form  of  a  query  language 
to  that  of  a  particular  application. 


2.2  Reply 

Reply  deals  with  informing  the  user  of  the  result  of  an  action.  The  result  can  be  the 
answer  to  a  query  or  information  about  the  status  of  a  request.  Presenting  output  to 
the  user  should  be  done  in  a  way  that  is  palatable  to  the  user  and  easy  for  him  to 
assimilate.  Again,  some  form  of  "language"  is  required  to  communicate  the  output. 
Displayed  output,  either  as  tables,  graphs,  pictures,  etc.,  seems  to  be  the  natural 
choice  for  videotex  systems  with  possibly  sound  also  available.  Five  characteristics  for 
representing  replies  are  important  for  evaluating  interactive  query  languages. 

1.  Presentation  complexity  is  a  measure  of  how  long  it  takes  the  user  to  grasp  the 
content  of  a  reply. 

2.  Multi-media  is  a  measure  of  the  number  of  ways  in  which  the  system  allows  people 
to  communicate. 
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3.  Customizing  of  replies,  as  for  customizing  of  requests,  means  that  the  form  of  the 
reply  is  determined  by  the  environmental  factors  of  the  interaction. 

4.  Dynamic  control  is. a  measure  of  the  ability  of  the  user  to  control  the  pace  at  which 
and  form  in  which  the  reply  to  a  request  is  presented  to  him. 

5.  RsiLsabilUy  of  a  reply  is  a  measure  of  whether  the  reply  to  a  request  is  only  for 
immediate  consumption,  or  may  be  useful  at  a  later  time  as  input  in  the 
formulation  of  another  request. 


2.3  Dynamics 

Dynamics  deals  vrith  the  nature  of  the  interaction  between  the  user  and  the  system. 
The  request  and  reply  processes  must  be  accomplished  via  some  communication 
medium  between  the  user  and  the  system  (i.e.,  the  "languages"  must  have  some  means 
of  transport  between  the  two  "participants").  For  example,  the  interaction  can  be  by 
typing,  pointing,  speaking,  etc.  The  quality  of  this  interaction,  from  the  user’s 
viewpoint,  is  a  critically  important  aspect  of  the  querying  process.  In  videotex 
systems,  we  would  like  this  interaction  to  be  as  easy  and  interesting  for  the  user  as 
possible.  Five  characteristics  of  the  interaction  between  the  user  and  the  system  are 
important  for  evaluating  interactive  query  languages. 

1.  The  bandvjidth  of  interaction  is  a  measure  of  the  speed  at  which  the  user  and  the 
system  communicate  with  each  other. 

2.  Gamesmanship  is  a  measure  of  how  challenging  and  interesting  the  interaction 
between  the  user  and  the  system  is. 

3.  Protocol  complexity  is  a  measure  of  the  number  of  protocols,  and  their  difficulty, 
available  to  the  user. 

4.  Responsiveness  is  a  measure  of  how  fast  the  system  responds  to  a  user’s  request. 

5.  Control  is  a  measure  of  how  comfortable  people  feel  in  using  a  system. 

In  most  interactive  query  languages,  these  three  issues  are  lumped  together.  This 
is  a  mistake  since  a  language  used  for  formulating  requests  has  different  user 
requirements  than  a  language  that  is  used  for  presenting  replies.  In  addition,  the 
dynamics  of  a  language  are  independent  of  the  static  aspects  of  formulating  requests 
or  presenting  replies.  ’Thus  requirements  for  the  dynemics  of  a  query  language  can  be 
considered  independent  of  request  and  reply  requirements. 


3  QUERYING  REQUIREMENTS 

In  the  previous  section,  we  divided  the  querying  process  into  three  parts:  request, 
reply  and  dynamics  of  interaction.  As  far  as  the  user  interaction  is  concerned,  the 
most  difficult  of  these  three  parts  is  the  request  part  and  its  dynamics.  The  reason  for 
this  is  that  in  forming  a  request  the  user  is  trying  to  inform  the  system  precisely  of 
what  he  wants.  'The  user  must  provide  the  system  with  a  great  deal  of  guidance  and  be 
very  exact  in  specifying  a  request  if  the  system  is  to  understand  it  correctly.  Replies 
from  the  system  on  the  other  hand  are  much  easier  for  the  user  to  understand  even  if 
they  are  presented  badly.  This  is  because  humans  are  much  more  intuitive  end 
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adaptable  them  computer  systems.  This  is  not  to  say  that  reply  aspects  of  quer3dng  are 
not  important.  However,  for  the  rest  of  this  paper  we  will  concentrate  mainly  on  the 
request  aspects  of  a  query  language  and  its  dynamics. 

In  general,  a  user  wishing  to  access  an  external  data  base  is  not  a  typist  and 
probably  does  not  want  to  become  one  to  use  the  query  lemguage.  Hence,  there  should 
be  a  minimum  number  of  keystrokes  required  to  communicate  with  the  system.  In 
order  to  meet  this  requirement  it  is  essential  that  some  type  of  pointing  mechanism  be 
available  for  pointing  at  things  on  the  screen.  The  pointing  method  should  be 
mechanical  (e.g.,  a  joystick  or  mouse)  so  that  the  user  is  free  to  concentrate  on  the 
screen  while  manipulating  the  pointing  device.  Most  likely,  it  will  be  impossible  to 
eliminate  all  typing.  Therefore,  to  be  least  tedious,  it  is  also  essential  that  typing 
mistakes  be  allowed  and  corrected  by  the  system  and  that  the  user  be  able  to  assign 
aliases  to  refer  to  objects  in  the  external  data  bases. 

Any  commands  that  are  required  to  direct  the  system  to  perform  some  action 
should  be  provided  via  function  keys  or  menus  of  operations  and  be  named  to  reflect 
their  functionality.  In  addition,  the  number  of  such  commends  should  be  kept  to  a 
minimum.  In  this  way  the  user  is  not  required  to  remember  the  commands  available 
and  can  easily  recall  their  function  from  their  names. 

The  system  should  guide  the  user  in  formulating  his  requests.  To  this  end  it  should 
display  options  available  and  also  provide  instructions  on-line  at  any  point  during  an 
Interaction.  Ihe  way  in  which  requests  are  formulated  should  be  intuitive  to  the  user. 
In  this  respect,  there  should  be  a  paradigm  for  formulating  requests  that  the  user  can 
easily  understand  and  that  helps  him  remember  the  interaction  protocols.  In  addition, 
the  system  can  provide  cues  to  the  user  (e.g.,  by  the  form  of  the  display)  that  help  the 
user  formulate  correct  requests. 

While  providing  guidance  to  the  user  in  formulating  his  requests,  we  must  be  careful 
not  to  put  the  user  in  a  straightjacket  by  completely  predetermining  the  requests  that 
Ccin  be  formulated.  Tlie  query  language  must  still  allow  the  user  some  latitude  in 
determining  the  selectivity  of  his  requests.  To  this  end.  the  query  language  must  at 
least  allow  the  user  to  specify  the  contents  of  his  reply  if  not  its  exact  format. 

In  most  query  languages  the  user  is  supposed  to  know  the  structure,  called  the 
schema,  of  the  data  base  he  is  accessing.  For  external  data  bases,  we  Ccinnot  expect 
the  user  to  know  or  even  understand  the  structure  of  the  data  base  or  the  difference 
between  the  schema  and  the  data  base.  Instead,  there  should  be  a  mechanism  to  guide 
the  user  through  the  structure  part  to  the  data  peirt.  In  addition,  the  way  in  which  the 
user  queries  the  structure  of  a  data  base  should  be  as  similar  as  possible  to  the  way  he 
queries  the  data  base  itself.  In  this  way  the  user  only  needs  to  learn  one  set  of 
interaction  protocols  that  apply  to  all  his  interactions  with  the  system. 

Studies  have  shown  that  users  of  computer  systems  can  become  very  lost  if  they  do 
not  know  where  they  are  in  the  system  [Memtei,  1962].  In  addition,  a  user  cannot  be 
expected  to  remember  everything  that  has  transpired  during  an  interaction.  To  help 
orient  the  user,  we  should  retain  on  the  screen  as  much  as  possible  the  route  we  have 
followed  in  querying  the  data  base  (or  at  least  have  it  available  for  display). 

User  interfaces  for  querying  nonformatted  data  are  often  very  different  from  those 
that  query  formatted  data.  When  querying  external  data  bases,  we  may  have  many 
different  types  of  data  available.  Most  data  will  probably  be  in  such  a  form  that  a  user 
cannot  distinguish  formatted  data  (e.g.,  inventory  information)  from  nonformatted 
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data-  (e.g.,  catalogue  sales  information).  In  addition,  it  is  probably  not  desirable  to 
require  a  different  set  of  protocols  for  different  types  of  data.  Thus,  the  user  interface 
should  be  as  uniform,  as  possible  for  querying  the  different  types  of  data  found  in 
external  data  bases. 

Even  though  we  provide  uniform  interaction  protocols  for  all  types  of  requests,  we 
can  allow  a  very  nonlinear  user  interface  in  terms  of  difficulty  of  request  formulation. 
That  is,  simple  requests  are  extremely  easy  to  formulate  while  more  elaborate  requests 
can  be  very  complex.  In  fact,  joins  and  other  complex  requests  can  be  assumed  not  to 
be  frequent.  The  user  interface  should  exIIow  these  queries  through  a  liigher  level, 
expertise  interface  which  can  access  the  same  data  as  the  interface  for  naive  users. 

The  user  interface  should  use  to  advantage  the  ability  of  people  to  remember  and 
easily  recognize  abstract  patterns.  For  example,  people  axe  much  better  at  processing 
image  data  than  are  computers.  In  addition,  they  readily  associate  abstract  images, 
such  as  logos,  with  the  objects  which  the  images  identify.  One  should  make  use  of  the 
image  processing  euid  associative  capability  of  people  in  a  query  language. 

Finally,  the  available  technology  (graphics,  colour  and  sound)  should  be  used  to 
distinguish  between  different  types  of  things  on  the  screen.  For  example,  we  could  use 
different  colours  to  distinguish  between  the  data  that  the  user  is  searching  for,  is 
providing  for  selection  or  is  provided  by  the  system.  Although  we  will  not  elaborate 
further  on  this  aspect  in  this  paper,  it  can  be  very  important  for  guiding  the  user  in  his 
interactions  with  the  system  and  should  be  carefully  considered  in  the  design  of  the 
screen  layout. 


4  PARADIGM 

The  paradigm  that  we  choose  for  accessing  Information  via  an  interactive  query 
language  for  external  data  baises  is  that  of  navigating  a  personal  spaceship  through 
space.  Our  spaceship  is  of  the  latest  design  and  is  very  powerful.  It  can  travel  both 
long  distances  (e.g..  those  between  galaxies)  and  short  distances  (e.g.,  those  between 
nearby  planets)  with  relative  ease.  Our  usual  objective  in  travelling  through  space  is  to 
arrive  at  a  planet  and  explore  it,  although  we  may  just  want  to  explore  space  per  se. 
Once  we  arrive  at  a  planet,  we  can  use  our  spaceship  to  explore  it  by  flying  around  or 
by  landing  and  examining  an  area  in  more  detail.  Space  is  very  vast  and  complex,  and 
usually  unfamiliELT  to  us.  Our  spaceship  is  equipped  with  a  view  screen  which  can  show 
us  a  visual  picture  of  space  in  any  direction,  but  only  a  limited  portion  of  it  at  a  time. 
It  is  very  difficult,  in  general,  to  navigate  without  aids  (i.e.,  visually  only)  although  when 
we  are  exploring  an  area  of  a  planet  this  may  be  desirable.  To  guide  us  in  our  travels, 
our  spaceship  can  provide  us  with  detailed  maps.  These  maps  come  in  different  levels 
of  abstraction  from  those  showing  the  general  structure  of  a  galaxy  to  those  sho^Nlng 
the  general  structure  of  a  planet.  These  maps  can  be  displayed  on  the  view  screen  in 
the  spaceship. 

In  a  similar  fashion,  we  can  think  of  the  information  sources  that  we  wish  to  access 
as  residing  in  a  vast  inforTriation  space.  The  composition  of  this  information  space  is 
patterned  after  the  composition  of  space  itself.  It  is  composed  of  abstract  objects, 
such  as  galaxies  and  star  systems,  and  concrete  objects  such  as  planets.  In  the  case  of 
the  information  space,  the  abstract  objects  correspond  to  the  structure  part  (schema) 
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of  the  data.  The  structure  part  tells  us  something  about  the  nature  of  the  data  we  can 
access.  The  concrete  part  corresponds  to  the  data  part  (data  base)  of  the  information 
space.  The  data  part  Is  the  smallest  level  of  detail  that  we  can  explore  in  the 
information  space.  Using  our  spaceship  (the  computer  system)  we  can  explore  the 
Information  space.  Our  spaceship  command  post  in  this  case  is  provided  by  the  query 
language  and  our  view  screen  is  provided  by  the  CF2T  (TV)  screen  through  which  we  view 
the  information  space.  Just  as  we  can  only  see  a  portion  of  space  at  a  time  on  our 
spaceship  view  screen,  so  we  can  only  see  a  portion  of  the  information  space  at  a  time 
on  our  CRT  screen. 

We  note  two  points  about  this  paradigm.  First,  we  distinctly  separate  the  structure 
part  from  the  data  part  conceptually,  but  operationally  (as  we  will  see)  they  are 
handled  (almost)  the  same.  This  separation  could  be  important  in  the  future  as  the 
structure  part  could  reside  in  the  user’s  terminal  much  like  the  maps  in  the  paradigm 
reside  in  the  spaceship.  The  data  part  exists  at  an  information  source  and  to  access 
and  explore  it  the  user  has  to  "go  to"  the  information  source  just  as  he  has  to  go  to  a 
planet  to  explore  it  in  detail.  Second,  the  way  in  which  we  have  described  the  user 
interaction  with  the  system,  most  of  the  details  of  how  to  access  information  are 
automatic.  The  user  supplies  certain  guidance,  but  the  system  worries  about  the 
details  of  how  to  do  things.  This  mode  of  operation  is  preferred  for  the  novice  user,  but 
may  be  tedious  for  the  experienced  user.  Therefore,  we  could  provide  a  manual 
override  to  the  user  and  let  him  guide  the  system  more  directly,  much  like  we  could 
provide  a  manual  override  in  our  spaceship.  This  corresponds  to  the  option  of 
providing  a  more  powerful  and  complex  query  language  for  the  experienced  user.  In 
this  paper  we  only  discuss  the  "automatic"  query  language,  but  the  possibility  of  a 
"manual"  query  language  as  a  complement  to  the  former  cilso  needs  to  be  investigated. 


5  DESIGN  OVERVIEW 

In  this  section  we  describe  the  overall  design  of  the  query  language.  We  discuss 
mhat  can  be  done  and  not  how  to  do  it.  Thus  we  describe  the  operations  generically 
and  use  generic  terms  such  as  "point  at",  "fill  in"  and  "select"  without  discussing  how 
these  operations  are  done.  We  use  the  analogy  developed  in  the  previous  section  to 
explain  the  operations  in  the  query  language.  In  the  next  section,  we  consider  how 
these  operations  can  be  done  by  the  user. 


5. 1  Choosing  an  External  Data  Base 

Suppose  that  we  are  in  our  personal  spaceship  somewhere  out  in  space  and  would 
like  to  choose  a  destination  to  go  to.  Using  our  view  screen,  we  can  scan  space  and 
look  at  the  stars.  However,  all  that  we  see  by  doing  this  is  an  abstract  pattern.  If  we 
are  experienced  space  pilots,  we  may  be  able  to  choose  a  destination  from  the  star 
patterh  directly  and  direct  our  spaceship  to  go  to  it.  However,  in  general  we  probably 
will  not  be  familiar  with  space.  To  choose  a  destination  for  our  spaceship,  we  can  scan 
different  parts  of  space  and  look  at  maps  of  each  part.  We  can  ask  to  look  at  only 
certain  parts  of  space  that  have  some  specified  properties  or  we  can  look  at  all  parts  of 
space.  As  we  look  at  the  maps,  we  can  ask  for  a  more  detailed  map  of  a  specific  region 


An  Interactive  Query  Language  for  External  Data  Bases 


8 


of  the  current  map.  While  looking  at  the  maps,  we  may  mark  some  regions  of  a  map  for 
future  reference  as  being  of  interest  and  requiring  a  second  look.  Eventually,  we  will 
decide  on  a  destination  and  direct  our  spaceship  to  go  there. 

Consider  now  trying  to  access  external  data  bases.  Y>Taen  we  first  sign  on,  we  are 
like  the  person  in  the  spaceship.  We  are  out  in  space  and  in  the  view  screen  (TY 
screen)  we  can  see  an  abstract  pattern  (template)  of  the  data  bases  that  we  can.  select 
from.  The  image  that  we  see  is  like  the  view  of  space  that  just  shows  us  the  star 
patterns  'svithout  any  information  about  what  they  represent,  For  example,  we  can  be 
presented  with  a  template  in  the  view  screen  such  as  that  shown  in  figure  1, 


EODTE; 


VIDEOTEX  INFORMATION  SERVICE  **** 


CATEGORY 


CONTENTS 


HESSAGE 


Figure  I  Empty  display  template. 

The  exact  format  of  the  template  in  the  view  screen  is  not  important  here  (this  is 
subject  to  study  to  determine  the  best  layout).  The  important  point  to  note  is  that  the 
template  is  divided  into  three  basic  parts.  One  part  specifies  the  pathvfe  have  taken  to 
2irrive  at  the  current  template  in  the  view  screen.  The  ROUTE  field  in  the  example 
contains  this  information.  A  second  part  is  used  for  the  specification  and  display  of 
data.  In  the  example  the  CATEGORY  and  CONTENTS  fields  serve  this  purpose.  Finally, 
there  is  a  part  used  for  system  feedback  to  the  user,  labelled  MESSAGE  in  the  example. 

If  we  have  no  idea  which  external  data  base  we  would  like  to  access,  we  can  ask  the 
system  to  display  all  the  information  it  has  about  data  bases  at  this  level.  In  this  case 
we  get  a  display  of  all  entries  that  correspond  to  the  template.  For  example,  we  might 
get  a  display  such  as  that  shown  in  figure  2  if  we  initially  ask  for  a  display  of  ail  entries. 
We  now  can  examine  the  information  map,  marking  some  entries  for  future  reference 
by  pointing  at  them  or  selecting  one  by  pointing  at  it.  Selecting  an  entry  allows  us  to 
look  at  more  detailed  information  about  it.  For  example,  selecting  the  category 
'entertainment'  and  requesting  a  display  of  all  its  entries  would  give  us  a  display  such 
as  that  shown  in  figure  3. 

Alternatively,  we  may  know  the  kinds  of  external  data  bases  we  are  interested  in, 
but  not  where  they  are.  In  this  case,  we  can  ask  the  system  to  show  us  more 
information  about  these  destinations  by  pointing  at  and  filling  in  the  CATEGORY  field. 
For  example,  we  may  fill  in  the  value  'entertainment'  in  the  CATEGORY  field  and  ask  for 
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ROUTE: 


♦♦♦♦VIDEOTEX  INFORMATION  SERVICE  •*♦♦ 


CATEGORY 

Entertainment 


News 


Restaurants 

MESSAGE:  Frame  1  of  10 


CON^TENTS 

Films 

Live  arts 

Exhibits 

Sports 

Television 

World 

Natiorxal 

Local 

Sports 

Entertainment 

Weather 

Chinese 


Figune  2  Nor^elective  display  of  all  available  information  sources. 


a  display  of  its  contents  as  in  figure  4  Subsequently  we  can  examine  the  information 
map.  mark  some  entries  for  future  reference  and/or  select  one  by  pointing  at  It.  For 
example,  if  we  select  the  'entertainment'  entry  and  subsequently  ask  for  a  display  of 
Its  entries  we  get  the  display  shown  in  figure  3. 

•♦♦♦VIDEOTEX INFORMATION  SERVICE  ♦♦♦♦ 


ROUTE:  Entertainment 


CATEGORY 

CONTENTS 

Films 

First  run 

Revues 

Festivals 

Live  arts 

Theatre 

Ballet 

Opera 

Recitals 

Exhibits 

Art 

Books 

Crafts 

Sports 

Hockey 

Soccer 

MESSAGE:  Frame  1  of  2 


Figure  3  Nonselective  display  of  Entertainment  information  source. 


Rather  than  explicitly  entering  a  field  value,  we  can  fill  in  a  field  value  by  pointing  at 
the  field  and  requesting  a  display  of  edl  the  field  values.  We  can  then  mark  some 
entries  as  being  the  values  we  want  to  fill  in  for  this  field.  In  this  case  rather  than 
entering  a  value  explicitly,  we  do  it  implicitly  by  marking  some  values  from  those 
displayed.  Thus,  for  example,  we  could  specify  the  display  shown  in  figure  4  by  pointing 
at  and  requesting  a  display  of  the  values  in  the  CATEGORY  field,  marking  the  entry 
’entertainment'  and  requesting  a  display  again. 

Finally,  if  we  know  where  we  want  to  go.  then  we  can  specify  this  directly  and  the 
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VIDEOTEX  INFORMATION  SERVICE 


ROUTE; 


CATEGORY 


CONTENTS 


Entertainment 


Films 
Live  arts 
Exhibits 
Sports 
Television 


HESSAGE:  Frame  1  of  1 

figure  4  Selective  display  of  available  information  sources. 

system  will  take  us  there,  We  specify  this  information  by  pointing  at  and  filling  in  a 
value  for  the  ROUTE  field.  This  directs  the  system  to  a  certain  destination  and  causes 
it  to  show  us  a  template  of  the  destination  selected. 

By  applying  this  technique  repeatedly,  we  can  navigate  the  structure  of  a  data  base 
to'arrive  at  the  data  of  interest.  Every  time  we  select  something  in  a  template  we  get 
more  detailed  information  about  the  route  to  a  data  base.  We  edso  retain  information 
about  the  path  we  chose  to  get  to  the  current  template.  This  is  analogous  to  trying  to 
determine  a  destination  for  our  spaceship.  This  technique  can  be  carried  out  to 
several  levels,  but  it  is  desirable  to  keep  the  number  of  levels  small  to  reduce  the 
amount  of  interaction  required  to  get  to  the  data.  In  general,  the  number  of  levels 
required  will  be  determined  by  the  complexity  of  the  data  base  being  accessed.  This 
aspect  is  in  line  with  our  stated  goal  of  relating  complexity  of  access  to  complexity  of 
the  iniormation  or  type  of  request  formulated. 

By  means  of  the  CATEGORY  and  CONTENT  fields,  we  in  effect  provide  a  keyword  and 
cross  referencing  capability.  That  is,  all  data  bases  related  to  a  specified  category  or 
categories  will  be  displayed.  As  well,  some  information  may  apply  to  more  than  one 
category.  Note  that  the  displayed  information  may  be  at  different  levels  in  the 
structure  part.  That  is,  some  words  may  refer  to  data  bases  themselves,  while  others 
may  refer  to  parts  of  a  data  base.  However,  this  is  irrelevant  to  a  user  and  he  need  not 
be  aware  of  this  distinction.  What  appears  in  the  view  screen  is  a  function  of  how  the 
cross  references  to  data  have  been  designed  within  the  system. 

Associated  with  each  keyword  displayed  in  the  view  screen  is  a  more  detailed 
description  of  the  information  represented  by  the  keyword.  At  any  point  in  time  we 
can  point  at  a  keyword  and  ask  for  a  description  of  it.  This  is  analagous  to  consulting  a 
guidebook  associated  with  a  map.  We  only  do  so  when  we  want  to  know  more  detailed 
Information  about  a  point  of  interest. 

The  ROUTE  field  allows  us  to  see  the  path  we  have  taken  to  arrive  at  our  current 
location.  At  any  time  we  can  backtrack  by  pointing  at  the  level  (name  in  the  path)  we 
want  to  go  back  to. 
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5.2  Exploring  the  Data 

Our  eventual  destination  in  our  spaceship  is  a  planet.  Once  we  arrive  at  a  planet,  we 
can  fly  around  and  look  at  it  or  we  cein  land  and  explore  a  small  area  of  the  planet. 
Again  to  help  us  navigate,  we  require  maps  of  the  planet.  These  maps  show  us  the 
abstract  structure  of  the  planet  but  no  detail  about  it.  By  flying  around  and/or  landing 
we  can  explore  the  planet  in  detail. 

In  a  similar  manner,  we  eventually  reach  the  data  part  of  a  data  base.  As  in 
exploring  and  navigating  the  structure  part,  we  can  now  explore  and  navigate  the  data 
part.  As  for  the  structure  part,  we  again  are  presented  with  an  abstract  pattern 
(template)  of  the  data  In  the  view  screen.  An  example  of  such  a  template  is  shown  in 
figure  5. 

•♦♦♦VIDEOTEX  INFORMATION  SERVICE  ♦♦♦♦ 

ROUTE:  Entertainment. Films. First  run 

FILM  STARRING  THEATRE  TIMES  PHONE# 


MESSAGE: _ 

FTg\ire  5  Empty  display  template  for  information  so\irce  Entertainment. Films.First  run. 

Specific  template  entries  are  selected  by  pointing  at  and  filling  in  a  field.  For 
example,  to  display  all  films  starring  Donald  Sutherland,  we  would  point  at  the 
STARRING  field  and  enter  the  value  ’Donald  Sutherland’.  The  system  would  then  display 
all  films  starring  Donald  Sutherland  that  are  showing  currently.  Alternatively,  we  could 
fill  in  a  field  by  pointing  at  it  and  asking  the  system  to  display  all  values  that  it  has  for 
that  field.  We  can  then  mark  certain  field  values  by  pointing  at  them.  The  system  will 
then  show  us  a  complete  display  of  edl  entries  in  which  that  field  value  appeairs.  For 
example,  we  could  ask  that  the  system  show  us  edl  values  for  the  STARRING  field.  We 
then  choose  those  values  of  interest  to  us  for  further  inspection. 

If  we  are  just  interested  in  browsing,  then  we  can  ask  that  all  template  entries  be 
displayed.  Subsequently,  we  can  mark  specific  entries  for  further  examination  by 
pointing  at  them.  This  action  has  the  effect  of  eliminating  those  entries  that  are  not  of 
Interest  to  us  and  thus  reducing  the  size  of  the  data  display. 
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5.3  Predefined  Routes 

As  we  travel  around  in  our  spaceship  we  soon  discover  that  there  are  certain 
destinations  that  we  go  to  very  often.  We  become  familiar  with  the  routing  to  these 
destinations  and  soon  do  not  require  maps  to  help  us  get  there.  We  can  specify  the 
destination  precisely  by  visual  navigation. 

In  a  similar  manner,  when  accessing  external  data  bases  there  will  be  certain  data 
that  we  access  very  frequently.  To  speed  up  our  routing  to  this  data,  we  can  predefine 
the  route  that  the  system  should  take.  We  can  name  this  route  and  then  specify  it 
explicitly  in  the  ROUTE  field.  This  is  analagous  to  pointing  to  our  destination  directly 
on  the  visual  star  patterns  that  we  see  from  our  spaceship.  These  predefined  routes 
are  appended  to  the  structure  part  of  the  videotex  information  service,  but  are  only 
available  to  the  user  who  defined  the  routes.  The  destination  of  the  routing  can  be  a 
point  in  the  structure  part,  data  part  or  part  of  the  data  part  (i.e.,  a  subset  of  the 
data).  In  this  way  the  output  that  the  user  receives  can  be  customized  to  some  extent 
by.  for  example,  filtering  out  certain  information  in  the  data  part.  Predefined  route 
names  can  be  distinguished  from  regular  data  base  names  by  highlighting  them  in 
some  suitable  manner. 


6  DESIGN  CONSIDERATIONS 

In  the  previous  section  we  discussed  what  the  query  language  can  do,  but  no  details 
were  provided  about  how  it  might  do  it.  We  used  generic  terms  such  as  "point  at",  "fill 
in"  and  "select"  without  specifying  how  these  actions  might  be  done.  In  this  section  we 
elaborate  on  how  some  of  the  actions  discussed  in  the  previous  section  might  be  done 
by  a  videotex  user. 

6. 1  Pointing 

One  of  most  fundamental  actions  required  in  the  interaction  protocols  discussed  in 
the  previous  section  is  the  ability  to  point  at  something  within  a  view  screen  for  the 
purpose  of  marking  or  selecting  it  for  further  actions.  Ideally,  this  pointing  should  not 
distract  the  user's  attention  from  what  is  being  displayed  in  the  view  screen.  Voice 
directed  pointing  would  be  highly  desirable,  but  perhaps  not  technically  or 
economically  feasible  currently.  The  next  best  pointing  mechanism  would  be  a 
mechanical  pointing  device  such  as  a  joystick,  puck  or  mouse.  The  user  is  then  able  to 
position  a  cursor  to  the  appropriate  position  within  the  view  screen  and  to  indicate  the 
selection  of  this  position^ 

We  generally  need  to  be  able  to  specify  a  field  in  a  template  and  perhaps  an  entry 
within  a  field.  For  generality,  these  two  specifications  should  be  independent  of  each 
other.  The  exact  semantics  of  the  specification  will  depend  on  the  characteristics  of 
the  template  displayed  in  the  view  screen.  The  ability  to  select  a  specific  field  and  a 
specific  entry  leads  to  the  notion  of  a  current  field  and  entry.  Visually  this  situation 
can  be  indicated  by  highlighting  the  current  entry  and  field  in  some  way  (e.g.,  blinking. 


1.  Note  that  it  is  neccesary  to  both  position  the  cursor  and  to  indicate  that  this  is  the  position  desired. 
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inverse  video  or  a  certain,  colour). 

Some  functions  (as  defined  below)  require  a  field  as  a  parameter.  If  no  field  is 
explicitly  specified,  then  the  default  field  is  the  entire  template.  Thus,  display  mode 
with  no  field  explicitly  specified  refers  to  a  display  of  all  the  values  associated  with  a 
template,  while  display  mode  following  the  selection  of  a  field  refers  to  a  display  of  only 
the  vedues  ajssoclated  vrith  the  selected  field.  If  the  current  field  is  the  template,  then 
an  entry  refers  to  all  the  values  that  define  one  instance  of  the  template. 

6.2  Positioning; 

In  our  discussion  we  have  not  considered  any  restrictions  on  the  size  of  the 
template  that  could  be  displayed  in  the  view  screen.  We  implicitly  assumed  that 
everything  could  be  displayed  at  once.  In  general,  however,  templates  may  be  larger 
than  the  view  screen  size.  Ideally,  we  would  like  to  be  able  to  move  the  view  screen 
continuously  across  the  template  as  if  it  were  a  TV  camera. 

In  videotex  systems,  the  information  space  is  usually  divided  into  fixed  size  units 
(pages)  which  are  the  size  of  the  view  screen  and  continuous  motion  in  all  directions  is 
not  possible.  Accordingly,  we  can  eissume  Instead  that  templates  are  divided  up  into 
fixed  size  fraTn.es.  A  frame  is  a  part  of  the  information  space  that  can  be  viewed 
through  the  view  screen  at  one  time  (much  as  one  views  a  photographic  slide  through  a 
slide  projector).  In  general,  frames  exist  in  all  directions  from  the  current  frame 
(rather  than  just  forward  and  backwards  as  when  viewed  through  a  slide  projector). 
Frames  above  and  below  the  current  frame  hold,  respectively,  previous  and  next 
template  entries.  Frames  to  the  right  and  left  of  the  current  frame  hold  additional 
data  related  to  the  current  frame  that  cannot  be  displayed  all  at  once  through  the  view 
screen.  Thus,  for  example,  tables  of  data  that  are  too  wide  to  fit  within  the  view  screen 
can  be  divided  up  into  several  vertically  adjacent  frames  (right  and/or  left  of  each 
other),  while  tables  of  data  that  are  too  long  to  fit  within  the  view  screen  can  be  divided 
up  into  several  horizontally  adjacent  frames  (above  and/or  below  each  other). 

To  position  the  view  screen  to  the  desired  frame,  there  is  a  framie  operation  with 
the  following  options  and  semantics: 

1.  frame  up  ^ moves  the  view  screen  to  the  freime  above  the  current  frame 

2.  frame  doum  —  moves  the  view  screen  to  the  frame  below  the  current  frame 

3.  frame  left  —moves  the  view  screen  to  the  left  adjacent  frame 

4.  frame  right  —moves  the  view  screen  to  the  right  adjacent  frame 

5.  frame  naim  —moves  the  view  screen  to  the  named  frame. 

Frames  can  be  named  symbolically  so  that  it  is  possible  to  position  the  view  screen 
on  them  directly.  In  most  cases  it  is  desirable  not  to  have  right  and  left  frames  as  this 
will  fragment  the  data  a  great  deal.  In  this  case  the  fraime  function  reduces  to  the  up, 
doxt/n  and  name  options. 

If  it  is  necessary  to  have  left  and  right  frames,  then  some  way  of  orienting  the  user 
as  to  his  current  position  is  desirable.  This  orientation  has  two  aspects  to  it.  First,  we 
want  to  tell  the  user  where  he  is  with  respect  to  the  entire  information  space  visible  at 
this  level.  One  technique  used  to  do  this  is  to  graphically  show  the  position  of  the  view 
screen  with  respect  to  the  entire  information  space.  For  example,  we  could  show  the 
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entire  information  space  as  a  polygon  and  the  view  screen  position  as  a  polygon 
superimposed  on  the  larger  poiygon  representing  the  information  space.  Alternatively, 
we  could  use  the  message  area  to  indicate  the  displacement  of  the  user  left  and/or 
right  of  some  reference  point  much  as  we  showed  the  displacement  vertically  in 
previous  examples. 

The  second  aspect  that  we  need  to  worry  about  is  maintaining  a  semantic 
connection  between  the  different  fraimes  in  the  view  screen.  In  the  vertical  direction 
this  is  not  a  great  problem  since  as  we  scan  up  and  down  the  same  template  or  the 
same  part  of  the  template  is  displayed  in  the  view  screen.  However,  in  the  horizontal 
direction  the  part  of  the  template  visible  in  the  vievr  screen  changes  as  we  move  left 
and  right.  It  is  therefore  desirable  to  keep  some  common  thread  between  horizontally 
adjacent  frames.  One  possibility  is  to  choose  one  of  the  template  fields  as  a  key  field 
and  to  keep  this  field  in  the  view  screen  at  all  times  as  we  move  horizontally.  For 
example,  in  figure  5  one  of  the  fields  of  the  template,  say  FILM,  could  be  kept  in  the 
view  screen  as  we  move  the  view  screen  position  horizontally. 


6.3  Control  Functions 

Control  functions  deal  with  signalling  the  system  that  it  should  note  something 
about  its  current  state  and  perform  some  action  based  on  the  state.  The  effect  of 
invoking  a  control  function  can  be  thought  of  as  invoking  a  procedure  that  takes  as  its 
input  the  current  system  state  and/or  field  value.  Depending  on  the  value  of  the  input 
parameter(s),  certain  actions  occur.  Control  functions  can  be  provided  in  one  of 
several  ways  (e.g.,  as  function  buttons  or  menu  commands). 


SELECT 

The  select  function  takes  as  its  input  parameter  the  current  field  value.  The  result  of 
the  function  depends  on  the  characteristics  of  the  field  value.  If  we  select  a  category 
field  value,  then  the  result  is  to  make  that  category  part  of  the  route  we  are  following. 
Selecting  a  name  in  the  ROUTE  field  causes  the  system  to  back  up  our  routing  to  that 
point.  As  we  will  see,  selecting  a  data  value  that  is  of  type  image  or  voice  results  in  the 
display  of  a  picture,  the  showing  of  a  film  or  videotape,  or  the  playback  of  recorded 
voice  messages.  If  a  field  value  has  no  explicit  procedure  defined  for  it,  then  selecting 
it  has  no  effect  (i.e.,  a  null  action  results). 

HARK 

The  mark  function  takes  as  its  input  parameter  the  current  field  value.  The  result  of 
the  function  is  that  the  system  notes  the  marked  value  for  future  reference.  For 
example,  when  we  are  examining  field  values  we  can  mairk  certain  ones  for  further 
action.  Marking  can  only  be  done  in  display  specification  mode. 
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DESCRIBE 

The  describe  function  takes  as  its  input  parameter  the  current  field  value.  The  result 
of  the  function  is  that  the  system  provides  a  detailed  explanation  of  the  current  field 
value.  This  explanation  can  be  visual,  verbal  or  both.  For  example,  for  a  category  field 
value  the  describe  function  can  be  used  to  get  an  explanation  of  the  information 
available  in  the  category.  Not  all  field  values  need  have  descriptions  associated  with 
them.  Therefore,  invoking  this  function  can  result  in  no  description  (i.e.,  a  null  action). 

HELP 

The  help  function  takes  as  its  input  parameter  the  current  system  state.  The  result  of 
invoking  the  help  function  is  to  receive  instructions  as  to  what  actions  can  be 
performed  by  the  user,  and  how,  given  the  current  system  state 

UNDO 

The  undo  function  taikes  as  its  input  parameter  the  current  system  state.  The  result  of 
the  function  is  to  undo  the  last  control  function  or  the  current  specification  mode. 

SUSPEND 

The  suspend  function  takes  as  its  input  parameter  the  current  system  state  plus 
optionally  a  user  supplied  name.  The  result  of  the  function  is  to  save  the  current  state 
of  the  user  interaction  A  suspended  user  interaction  can  be  reactivated  at  any  time. 
The  interaction  proceeds  from  the  point  of  suspension  as  if  it  had  never  been 
interrupted.  Suspended  user  interactions  are  kept  in  a  local  data  base  named 
’suspended’  and  can  be  reactivated  by  selecting  them. 

EXIT 

The  exit  function  taices  as  its  input  parameter  the  current  system  state.  The  result  of 
the  function  is  to  end  the  current  user  interaction.  No  information  about  the  current 
interaction  is  saved. 

SAVE 

The  save  function  takes  as  its  input  parameters  the  current  system  state  plus  a  user 
supplied  name.  The  result  of  the  function  is  to  save  the  specification  of  the  user 
interaction  and  to  name  it  explicitly.  Saved  Interactions  are  kept  in  a  local  data  base 
named  ’saved',  The  interaction  Ccin  subsequently  be  duplicated  by  selecting  the  user 
defined  name.  The  user  defined  name  becomes  part  of  the  structure  part  information 
for  this  user.  The  save  function  provides  a  view  definition  capability.  The  save  function 
differs  from  the  suspend  function  in  that  a  suspended  interaction  is  only  maintained 
temporarily  until  it  is  completed.  A  saved  interaction  on  the  other  hand  is  maintained 
permanently  by  the  system. 
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DROP 

The  drop  function  takes  as  its  input  parameter  the  current  field  value.  If  the  current 
field  value  is  a  saved  interaction,  then  the  result  of  the  function  is  to  drop  the  saved 
interaction  from  the  structure  part  information  for  this  user.  Dropping  a  non-save d 
interaction  results  in  a  null  action. 


S.4  Specification  Modes 

Specification  modes  deal  -with  the  way  in  which  a  user  specifies  requests  for 
information  to  the  system.  Invoking  a  specification  mode  implies  that  a  sequence  of 
actions  will  follow  to  form  the  specification.  A  specification  mode  is  ended  implicitly  by 
the  select  function,  save  function  or  another  specification  mode,  or  explicitly  by  the 
undo  function  or  exit  function.  When  interacting  with  external  data  bases,  there  are 
two  ways  that  the  user  can  specify  his  request.  One  of  these  will  be  the  default  mode; 
which  one  is  determined  by  the  user.  The  default  mode  is  entered  initially  when  the 
user  signs  on  and  after  every  select,  suspend,  save  or  drop  function. 

FILL 

FLU  mode  allows  the  user  to  select  various  fields  and  to  specify  a  value  or  values  for 
each  field  selected  [Zloof,  1975a, b,  1977].  Fields  are  selected  by  pointing  at  them. 
They  aire  filled  in  explicitly  by  entering  a  value  from  a  keyboard  or  implicitly  by 
marking  displayed  values  as  explained  under  display  mode. 

If  multiple  values  are  specified  for  a  field,  then  the  or  boolean  operation  is  assumed 
among  the  values  (i.e.,  match  where  value-1  or  value-2,  etc,).  Note  that  an  and  boolean 
among  values  does  not  malce  sense  unless  an  entry  within  a  field  in  fact  can  have 
multiple  values.  If  more  than  one  field  is  filled  in,  then  the  and  boolean  operation  is 
assumed  among  fields  (i.e.,  match  where  field-1  and  fleld-2,  etc.).  An  or  boolean  among 
fields  can  be  handled  by  filling  multiple  templates. 

The  end  of  template  filling  is  signalled  by  entering  display  mode  mth  the  current 
field  being  the  template.  The  user  is  then  shown  aU  entries  of  the  template  that  match 
the  specification.  If  no  fields  have  been  filled  in,  then  all  entries  corresponding  to  the 
template  are  displayed. 

DISPLAY 

Display  mode  allows  the  user  to  look  at  entries  of  fields  or  templates  in  the  view  screen. 
If  the  template  is  empty  when  display  mode  is  invoked,  then  all  entries  are  displayed. 
If  the  template  has  been  filled  in.  then  only  entries  that  match  the  filled  in  template 
are  displayed  as  discussed  under  fiU  specification  mode. 

In  display  mode  the  template  entries  can  be  scanned  using  the  pointing  and  view 
screen  positioning  operations.  The  current  field  and  entry  can  be  selected  using  the 
select  function  which  invokes  the  procedure  associated  with  the  current  field  value. 
Field  values  can  also  be  marked  in  display  mode.  Subsequently,  entering  display  mode 
again  will  select  only  marked  field  values  for  display.  If  the  current  field  is  the 
template,  then  marking  has  the  effect  of  selecting  only  the  marked  entries  for  display. 
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If  the  current  held  is  a  field  within  the  template,  then  the  effect  is  to  select  only  those 
entries  that  match  the  marked  field  values.  The  same  conventions  apply  in  this  case  as 
for  fill  mode. 

6.5  Data  Types 

Most  traditional  query  languages  handle  only  formatted  data  in  a  reasonable 
manner.  However,  there  are  other  t}’pes  of  data  that  may  need  to  be  dealt  with  when 
querying  external  data  bases.  Some  of  these  are:  formatted  text  data,  nonformatted 
text  data,  image  data,  and  voice  data.  Because  of  the  nature  of  the  different  t^pes  of 
data,  the  way  in  which  they  are  queried  is  specific  to  the  data  type.  However,  the  same 
interaction  protocols  can  be  used  on  all  types  of  data. 

FORMATTED  TEXT  DATA 

Once  we  reach  the  data  part  (instance  level)  in  the  quer3ing  process,  we  are  presented 
with  a  template  of  the  format  of  the  data  part.  An  example  of  such  a  template  is  shown 
in  figure  6. 


♦♦•♦VIDEOTEX  INFORMATION'  SERVICE 
ROUTE:  Entertainment. Films. First  run 

FILM  ^AREdliG  THEATRE  TIME  PHONE# 


MESSAGE:  _ — - 

Figure  6  Display  template  for  formatted  data. 

We  can  select  from  such  a  template  by  using  the  fill  and/or  display  specification 
modes  discussed  above.  The  default  for  field  filling  assumes  that  we  want  an  exact 
match  for  the  value  specified.  For  certain  types  of  fields  we  can  also  allow  other  kinds 
of  matches  such  as  pattern  matches  (find  a  certain  pattern  in  a  field),  less  than.  less 
than  or  equal,  greater  than,  greater  than  or  equal  and  not  equal  matches.  These 
options  can  be  indicated  by  prefixing  the  specified  value  with  a  suitable  symbol  (e.g.,  <. 

etc.)  [Zloof.  1977].  Fields  to  be  filled  can  be  selected  using  the  pointing  operation 
discussed  earlier. 
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NONFORHATTED ITEXT  DATA 

If  the  data  part  consists  of  nonformatted  text  data,  then  when  we  arrive  at  the  data 
part  we  are  again  presented  with  a  template  of  the  nonformatted  text  data.  An 
example  of  such  a  template  is  shown  in  figure  7. 

♦♦♦♦  VIDEOTEX  INFORMATION  SERVICE 
ROUTE:  Books. Novel. Gone  with  the  Wind. Chapter  1 


SIESSAGE _ 

figure  7  Display  template  for  nonformatted  data. 

We  can  select  from  text  by  specifying  a  pattern  for  which  we  are  looking  in  the  text 
[Tsichritzis  and  Chris todoulakis,  1982].  We  do  this  by  filling  in  the  text  pattern  in  the 
appropriate  field.  Displaying  the  specified  pattern  causes  the  system  to  show  us  the 
pattern  in  context  (i.e.,  where  it  appears  in  the  text).  In  this  case,  the  next  and 
previous  options  for  the  entry  operation  will  show  us  the  next  and  previous  entries  with 
the  pattern  in  context  (if  the  current  field  is  the  template).  We  can  also  look  at  other 
frames  around  the  current  frame  with  the  frame  operation.  Entering  display  mode 
without  form  filling  will  show  us  all  the  text. 

MAGE  AND  VOICE  DATA 

Certain  fields  within  formatted  or  nonformatted  text  data  may  correspond  to  image  or 
voice  data.  Currently  it  is  very  difficult  to  query  these  types  of  data  directly.  In  the 
future  it  may  be  possible  to  input  a  picture  or  a  voice  message  and  ask  the  system  to 
find  all  instances  of  them.  However,  for  now  it  is  only  possible  to  display  or  play  back 
the  image  or  voice  data.  Accordingly,  this  playback  or  display  action  can  be  initiated 
by  selecting  the  appropriate  entry  within  an  image  or  voice  field  by  the  select  function. 
In  order  to  indicate  to  the  user  the  nature  of  these  fields,  they  can  be  suitably 
highlighted  by  specific  colours  or  marked  by  special  graphic  symbols  (e.g.,  an  icon  of  a 
camera  to  indicate  photographic  pictures). 
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7  DISCUSSION  OF  THE  QUERY  LANGUAGE  DESIGN 

In  this  paper,  we  examined  the  process  of  querying  via  an  interactive  query 
language  in  general  and  with  respect  to  external  data  bases  in  particular.  We  first 
divided  the  querying  process  into  three  parts;  request,  reply  and  dynamics.  For  each 
part  we  identified  several  characteristics  and  indicated  some  desirable  properties  of 
these  characteristics.  We  then  proposed  a  design  for  a  query  language  for  external 
data  bases. 

In  developing  our  design,  we  concentrated  on  the  request  and  dynamics  aspects  of 
the  query  language.  For  request  the  pareimeters  of  user  interaction  were:  number  of 
keystokes  required,  type  and  number  of  commands  available,  degree  of  freedom  in 
request  formulation,  selectivity  of  requests,  uniformity  of  requests  with  respect  to  data 
bases  accessed  and  the  ability  to  customize  the  request  formulation  for  a  specific 
application.  For  the  dynamics  of  interaction  the  parameters  related  to:  the  bandwidth 
between  the  user  and  the  system,  the  degree  of  gamesmanship  involved  in  the 
Interaction,  number  and  difficulty  of  protocols  available  for  interaction,  the 
responsiveness  of  the  system  to  a  user  request  and  the  degree  of  control  experienced 
by  the  user  in  an  interaction.  Let  us  examine  the  proposed  design  with  respect  to 
these  properties. 

The  number  of  keystrokes  required  to  specify  a  request  depends  on  the 
specification  mode  chosen  by  the  user.  If  the  user  displays  and  marks  entries,  then  he 
need  only  point  at,  mark  aind  select  values.  Marking  requires  one  keystroke  for  each 
value  marked  while  selecting  also  requires  one  keystroke  for  each  value  selected.  If  fill 
mode  is  used,  then  the  user  may  have  to  point  at  fields  and  type  in  field  values. 
However,  even  in  this  case  many  keystrokes  can  be  avoided  if  the  user  displays  field 
values  and  marks  them. 

In  the  proposed  design  there  are  two  specification  modes,  nine  control  functions, 
one  pointing  operation  and  one  positioning  operation.  One  of  the  two  specification 
modes  will  be  the  default  mode  so  the  user  need  not  specify  it  explicitly.  Of  the  nine 
control  functions,  two  are  likely  to  be  used  heavily  —  select  and  mark.  If  the  control 
functions  are  implemented  as  function  buttons  and  labelled  with  the  function  names, 
then  there  will  be  no  need  for  the  user  to  remember  the  function  names.  In  addition, 
the  functions  are  named  so  as  to  clearly  convey  their  meaning  and  on-line  help  is 
always  available  via  the  help  function. 

Because  of  the  way  the  user  interaction  has  been  designed,  it  is  very  difficult  to 
formulate  a  request  incorrectly,  either  syntactically  or  semantically.  Syntactically, 
the  query  language  has  no  syntax  to  remember.  The  specification  modes  accept  any 
input  state  for  the  templates,  from  empty  to  fully  filled.  Control  functions  are  always 
valid  although  they  may  return  a  null  result.  Semantically,  field  names  always  appear 
in  the  view  screen  and  the  user  can  see  the  structure  of  the  data  via  the  template.  It  is 
possible  to  specify  a  complicated  request  incorrectly  (e.g.,  if  there  are  many  fields  to 
be  qualified  with  and's  and  or's).  However,  the  way  in  which  fields  on  a  template 
interact  by  default  (andl’s  between  fields,  or’s  within  a  field)  is  fairly  intuitive.  It  must 
be  accepted  that  complicated  requests  are  complex  and  therefore  prone  to  error  in 
their  specification. 

The  selectivity  of  the  query  language  is  almost  as  good  as  keyword  and  by  example 
query  languages.  The  only  difficult  operation  to  specify  is  a  join-like  operation  (in  fact 
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we  have  not  said  how  to  do  it*).  However,  we  do  not  feel  that  there  will  be  a  general 
need  for  a  join-like  capability  in  a  query  language  for  external  data  bases.  In  any  case, 
one  can  alw'ays  have,  the  option  of  going  to  a  more  experienced  user  interface  (i.e.,  a 
keyword  query  language)  for  such  requests. 

In  the  proposed  query  language,  we  did  not  make  any  assumptions  about  the 
structure  of  the  external  data  bases.  The  nature  of  the  interface  chosen  —  templates  — 
does  not  presuppose  any  particulcO'  underlying  structure  for  the  data  base.  Thus,  the 
query  lamguage  can  be  used  to  interface  to  any  type  of  data  base  provided  suitable 
translation  algorithms  are  provided  to  translate  the  operations  and  structures  in  the 
query  language  to  the  operations  and  structures  of  the  DBMS  of  the  external  data  base. 
This  is  currently  an  active  research  area  and  several  approaches  to  the  problem  can  be 
taken  [Date,  I960:  Vassiliou  and  Lochovsky,  1980]. 

The  query  language  can  be  easily  customized  to  particular  applications  by 
customizing  the  templates  via  which  the  user  interacts  with  the  system.  No  other 
aspect  of  the  query  language  need  change  to  accommodate  this  customizing.  In 
particular,  the  hardware  required  for  interaction  and  the  pointing  and  positioning 
operations,  control  functions  and  specification  modes  remain  the  same. 

The  bandwidth  of  interaction  achieved  depends  on  the  mode  of  specification  chosen. 
For  display  and  marking  it  can  be  very  high.  For  template  filling  it  can  be  moderate. 
Note  that  the  basic  concepts  behind  the  interaction  remain  unchanged  as  we  move 
from  pointing  via  a  keyboard  to  a  mechanical  device  to  voice.  Thus,  as  technology 
advances  allow  the  bandwidth  to  increase,  the  query  language  can  take  adveintage  of 
these  advances. 

The  gamesmanship  of  the  query  language  is  moderate  to  good.  The  user  can  easily 
explore  alternate  paths  in  the  structure  with  little  effort  since  he  can  easily  save  paths 
and/or  back  up  at  any  time.  This  exploration  process  gradually  allows  the  user  to 
move  toward  his  goal  and  he  can  see  his  progress  by  the  PJOUTE  field  in  the  view  screen. 
Depending  on  how  the  cross  references  are  constructed,  the  user  can  reach  the  same 
data  from  many  different  paths.  The  pointing,  marking,  displaying  and  filling  t3p)es  of 
activities  involve  the  user  in  the  interaction.  The  design  of  the  templates  and  the  use 
of  colour  and  sound  can  add  to  the  gamesmanship  of  the  query  language. 

There  are  several  protocols  for  querying  data  bases  —  fill,  display  and  predefined 
routes.  The  user  can  choose  the  one  he  feels  is  most  effective  for  his  particular 
interaction  and  skill  level.  He  can  also  set  the  default  specification  mode  to  suit  his 
knowledge  level  of  the  system. 

Responsiveness  may  be  a  limiting  factor  for  the  query  language  with  today's 
technology.  Display  mode  especially  requires  that  the  system  quickly  display  the 
requested  information.  When  we  are  at  the  data  part  level  this  can  mean  the  retrieval 
of  a  great  deal  of  information.  Being  able  to  display  the  values  of  any  field  implies  that 
there  is  some  fast  access  mechanism  available  to  quickly  retrieve  all  the  values.  The 
use  of  display  mode  may  have  to  be  limited  initially  to  the  structure  part  and  certain 
fields  of  the  data  part  to  overcome  the  response  problems.  However,  advances  in 
technology  may  soon  make  such  a  restriction  unnecessary. 

Finally,  the  entire  nature  of  the  user  interaction,  directing  the  system  to  display 
things,  mark  things,  etc.,  contributes  to  the  user's  feeling  of  control.  He  is  directing 


1.  One  possibility  is  to  use  a  saved  interaction  as  Input  to  another  interaction. 
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the  system  to  do  things  for  him  in  reaching  his  goal.  Also,  the  paradigm  for  querying 
introduced  earlier  helps  to  contribute  to  this  feeling  of  control.  The  paradigm  makes 
the  user  feel  that  the  system  is  his  friend  helping  him  guide  his  space  ship  (the 
system)  through  the  unfriendly  environment  of  space  (the  Information  space)  and 
protecting  him  from  it. 

8  SUMMARY 

The  advent  of  videotex  systems  emd  the  growth  in  communications  networks  are 
making  feasible  access  to  a  variety  of  computerized  information  sources  by  people  not 
familiar  with  computer  technology.  In  order  to  make  effective  use  of  these  information 
sources,  access  to  external  data  bases  will  have  to  be  via  an  easy  to  use  interactive 
query  language.  In  this  paper,  we  proposed  a  design  for  such  a  query  language. 

We  first  considered  the  process  of  accessing  a  data  base  (querying)  in  terms  of  the 
user  interaction  characteristics  of  the  process.  We  divided  the  querying  process  into 
three  parts:  request,  reply  and  dynamics.  For  aU  three  parts,  certain  characteristics 
were  identified  that  describe  the  parameters  of  the  user  interaction.  For  requests,  the 
parameters  are  number  of  keystrokes  required,  type  and  number  of  commands 
available,  degree  of  freedom  in  request  formulation,  selectivity  of  requests,  uniformity 
of  requests  with  respect  to  data  bases  accessed  and  ability  to  customize  the  request 
formulation  for  specific  applications.  For  replies,  the  parameters  are  the  form  in 
which  the  reply  is  presented  to  the  user,  whether  the  reply  can  be  presented  via  more 
than  one  medium,  whether  the  reply  can  be  customized,  the  degree  of  dynamic  control 
over  the  reply  and  the  possibility  of  saving  the  reply  and  using  it  as  input  at  some  later 
time.  For  the  dynamics  of  Interaction,  the  parameters  relate  to  the  bandwidth 
between  the  user  and  the  system,  the  degree  of  gamesmcinship  involved  in  the 
interaction,  number  of  protocols  available  for  interaction,  the  responsiveness  of  the 
system  to  a  user  request  and  the  degree  of  control  experienced  by  the  user  in  an 
Interaction. 

For  each  of  these  parameters  some  requirements  that  a  query  language  for 
external  data  bases  should  have  in  terms  of  a  desirable  ’Value"  for  the  parameter  were 
identified.  With  these  requirements  in  mind  we  then  proposed  a  design  for  a  query 
language  for  accessing  external  data  bases. 

For  describing  the  query  language,  a  paradigm  of  navigating  a  personal  spaceship 
through  space  using  detailed  maps  for  guidance  was  used.  Basically  the  user  is  able  to 
request  maps  of  the  information  space  available  to  him  and  to  choose  his  destination 
from  these  maps.  The  maps  are  presented  in  the  form  of  templates  which  show  the 
structure  and/or  contents  of  the  external  data  bases.  Most  of  the  interaction  is 
accomplished  by  pointing  at  fields  of  the  templates  and  either  fillmg  in  values  or 
requesting  a  display  of  vedues.  The  nature  of  the  interaction  allows  formatted  data, 
nonformatted  text  data,  image  data  and  voice  data  to  be  manipulated  in  a  umform 
manner. 

The  query  language  is  being  implemented  in  C  running  under  UNIX  on  a  VAX- 11/780. 
The  view  screen  Is  a  DEC  GIGI  colour  graphics  terminal.  The  terminal  allows  all 
graphics  processing  as  well  as  some  of  the  user /system  interaction  to  be  done  locally. 
Initially,  relational  data  bases  will  be  accessed,  but  the  capability  of  accessing  DBTG- 
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network  data  bases  will  also  be  added  eventually. 
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ABSTRACT 

Using  a  text  editor  as  a  paradigm  for  an  interactive  database 
query  and  update  language  is  proposed  Instead  of  editing  a  text 
file,  the  user  edits  an  underlying  relational  database.  Some  argu¬ 
ments  in  favor  of  such  a  language  are  given  and  a  prototype 
design  and  implementation  are  sketched. 


1.  Introduction 

Consider  an  interactive  text  editor  whose  command  for  finding  the  first 
line  in  the  current  file  containing  the  pattern  *AOM’  were  something  like  this: 

RANGE  OF  t  IS  current_file; 

DISPLAY  FIRST  t  WHERE  t  CONTAINS  'aom’; 

GO: 

It  is  likely  that  such  an  editor  would  not  prove  popular  with  frequent  users. 

When  we  look  at  existing  interactive  query  languages  for  database  systems, 
(see  [LT]  for  a  survey)  we  often  find  the  same  degree  of  verbosity  and  redun¬ 
dancy  as  in  the  example  above.  Two  reasons  are  usually  offered  for  this  type  of 
design.  First,  the  presumed  similarity  of  queries  to  natural  language  requests  is 
thought  to  malce  query  formulation  easier.  This  is  reminiscent  of  the  argument 
advanced  twenty  years  ago  that  COBOL  would  make  programming  in  English 
possible.  Second,  the  verbosity  of  the  language  is  designed  to  make  queries 
easier  to  read  and  thus  to  understand  and  maintain.  This  does  not  seem  a  cru- 
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cial  consideration  for  a  mainly  interactive  system,  vrhere  maintenance  is  non¬ 
existent.  The  reason  most  of  the  systems  mentioned  above  emphasize  readabil¬ 
ity  is  that  they  have  the  explicit  goal  of  providing  the  same  interface  to  the  pro¬ 
grammer  as  to  the  terminal  user.  In  other  application  areas,  this  goal  is  not 
deemed  achievable  or  indeed  desirable.  For  example,  a  programmer  and  an 
interactive  user  are  usually  provided  rather  different  methods  for  accessing 
the  operating  system  facilities. 

If  we  abandon  for  now  the  notion  of  a  common  interface,  we  are  faced  with 
the  question  of  how  to  design  a  database  interface  specialized  for  interactive 
access  and  update.  An  important  effort  in  this  direction  was  Query-By-Example 
[Z],  and  similar  form-oriented  systems  that  followed  it.  There  are  several  rea¬ 
sons  why  we  think  it  worthwhile  to  explore  adternatives  to  the  forms-based 
approach.  The  first  rather  mundane  problem  is  that  there  are  still  many  300- 
baud  terminals  in  the  world,  and  any  screen-oriented  interface  will  be  inap¬ 
propriate  for  them,  not  to  mention  printing  terminals.  Second,  although  our 
system  is  intended  for  interactive  use,  we  would  like  to  make  it  available  to  pro¬ 
grams  or  to  packaged  sequences  of  commands.  This  is  difficult  to  dd  for 
screen-based  interfaces,  wnicii  are  essentially  two-dimensional  and  require 


awkward  encodings  to  be  represented  in  linear  form  [IBM].  Finally,  we  do  not 
find  forms-based  systems  easy  to  use  in  the  browsing  mode  that  seems  to  us  an 
important  component  of  interactive  access. 

In  this  report,  we  propose  the  familiar  line-oriented  interactive  text  editor 
as  the  paradigm  on  which  to  base  a  query/update  database  interface.  In  partic¬ 
ular,  we  describe  a  preliminary  design  based  on  the  UNIX  ed  editor  [TR],  and 
influenced  also  by  the  awk  system  [AWTC]. 


I - ; - 

UNIX  is  a  trademark  of  Bell  Laboratories. 
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¥e  should  say  a  word  on  fuU-soreen  editors  versus  line  editors.  Fof  the 


same  reasons  mentioned  in  the  last  paragraph,  vro  have  decided  to  use  the  line 


editor  model,  (n  s^Le  of  Ihe  claims  lhal  line  edllors  are  fcy  hov  obidiele.  ft 


should  not  prove  difficult  In  the  future  to  graft  a  full  screen  interface  onto  our 
database  editor,  ahd  in  fact  this  is  the  way  many  current  fUll-screen  text  edi¬ 
tors  have  been  implemented.  On  the  other  hand,  we  would  like  to  consider 
more  caLrefuiiy  what  a  fua-sbreeft  database  editor  should  look  like  before  pro- 
posing  sucn  am  extension* 


The  maih  reason  for  choosing  a  text  editor  model  for  our  interface  is  that 
for  regular  users  of  an  interactive  computer  system,  the  editor  Is  normally  the 
most  familiar  and  frequently  used  tool.  The  task  of  retrieving  and  updating  data 


from  a  database  bah  in  fact  be  viewed  as  an  editing  process,  as  can  many  other 


data  maLnipuiations  performed  interactively  by  users.  Fraser  [F]  has  pointed  out 
the  desirability  of  having  a  common  user  interface  for  all  these  editing  tasks. 
The  conciseness  and  flexibility  of  text  editors  provides  a  nice  contrast  to  the 
structured  query  languages  mentioned  above.  A  mode  of  interaction,  in  which 


abservatidff  af  mail  PSgiSRs  sf  the  database  mtsfhates  witB  Large  juffips  aha 

global  feqUests  se§m§  well  sUitSd  tb  th§  editor  Siodel. 


The  tex±  editor  interface  is  mapped  to  the  database  context  in  a  straight¬ 
forward  way:  a  file  is  a  relation,  a  line  is  a  tuple.  The  concept  of  columns  or 
fields  has  no  analogue  in  text  editing  and  is  introduced  in  our  design  in  a  rather 
ad-hoc  fashion,  similsir  to  the  one  in  awk. 


The  rest  of  the  paper  sketches  a  description  of  the  database  editor.  The 
description  is  deliberately  vagUe  at  times,  since  this  paper  is  ihterlded  as  a 
preliminary  report  and  the  system  is  likely  to  undergo  substantial  changes  as 
we  gain  some  experience  with  iL 


Database  Editor 


26 


2.  iSffa 

The  database  editor  is  designed  to  act  on  a  relational  database.  There  is  at 
ail  times  a  current,  relation  and  a  current  tuple  within  the  current  relation.  The 
current  relation  is  established  when  invoking  the  editor,  for  example 

edb  /usr/dbase/cacm/bibMo 

As  a  running  example,  we  will  use  a  database  containing  a  bibliography  of 
the  Communications  of  the  ACM.  There  are  four  relations:  Papers,  Volumes, 
Authors,  and  Keywords,  abbreviated  P»  V,  A,  IC.  They  contain  the  following  attri¬ 
butes: 

P  (vol, issue, page, title) 

V  (vol, year) 

A  (vol, issue, page, author) 

K  (vol, issue, page,keywd). 

The  current  relation  can  only  be  changed  with  the  e  command,  as  in 

e  P 

All  editor  commands  refer  to  the  current  relation  and  the  current  tuple  unless 
the  user  explicitly  says  otherwise. 

The  cbiumns  of  a  relation  are  called  diinbuies  or  fields,  k  field  can  be 
designated  in  twn  ways:  by  giving  its  name  or  its  column  number.  A  field 
number  is  $i,  where  ?  is  a  number  between  0  and  the  number  of  fields  of  the 
relation.  The  field  number  is  relative  to  a  conventional  ordering  imposed  by 
the  underl3ring  database  system.  As  in  awk.  $0  denotes  the  whole  tuple.  Both 
forms  may  be  preceded  by  a  relation  name,  as  in  P.$2  ;  the  current  relation  is 
assumed  otherwise. 

The  structure  of  the  database  can  be  examined  by  the  f  command  (from 
the  UNIX  editor  ’file’  command).  This  will  display  the  relation  names  in  the 
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current  database  and  their  attribute  names,  with  the  current  relation  name 
starred  or  highlighted.  .  / 

3.  Displaying  Data 

Tuples  In  the  immediate  vicinity  of  the  current  tuple  can  be  displayed 
easily.  A  p  command  displays  the  current  tuple;  a  carriage  return  or  a  + 
advance  to  the  next  tuple  in  some  arbitrary  order  and  display  it;  a  -  moves  to 
and  displays  the  previous  tuple.  Any  of  these  can  be  followed  by  a  list  of  fields, 
indicating  that  only  those  fields  are  to  be  displayed,  in  the  order  in  which  they 
are  listed. 

For  more  selective  access,  predicates  are  used  to  specify  a  tuple  or  set  of 
tuples  to  be  displayed.  A  predicate  is  an  arbitrary  Boolean  combination  of  com¬ 
parisons  between  two  attributes  or  between  an  attribute  and  a  constant.  Con¬ 
stants  can  be  numbers,  strings,  or  regular  expressions.  For  example, 

a  A 

/$1  sFraser/ 

will  display  the  first  tuple  in  the  A  relation  with  the  value  "Fraser"  in  the 
Author  column.  The  meaning  of  "first”  here  is  implementation  dependent;  no 
particular  ordering  Is  assumed  (but  see  the  sorting  command  below).  The  next 
time  the  command  is  issued,  the  next  tuple  satisfying  the  condition  will  be 
displayed.  If  the  same  condition  is  required  repeatedly,  it  can  be  elided  by  //  as 
in  the  standard  UNIX  editor.  Again  as  in  ed,  using  ?  instead  of  /  as  a  delimitier 
for  the  predicate  will  cause  a  "backwards"  search. 

Regular  expressions  are  similar  to  those  in  ed  or  awk;  for  example, 

/vol<22  It  auth  ~  [FraserlAhojKernl]/ 

will  display  the  next  tuple  in  the  current  relation  (say  A)  where  the  volume 
number  is  less  than  22  and  the  author  column  matches  one  of  the  three  strings 
separated  by  |.  Note  the  use  of  the  comparator,  borrowed  from  awk  to  denote 
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regular  expression  matching. 

The  g  prefix  can  be  added  bo  a  display  command  to  cause  display  of  all 
tuples  satisfying  the  predicate: 

e  K 

g/keywd  ~  [sjS][ort|earch]/ 

A  screen  paging  mechanism  is  used  whenever  the  amount  of  output  exceeds  the 
screen  size  to  let  the  user  control  the  scrolling  speed, 

A  *  by  itself  in  the  predicate  slot  denotes  the  vacuous  predicate,  so 

Q/*l 

produces  a  complete  display  of  the  current  relation.  As  before,  a  list  of  field 
names  can  be  used  to  restrict  the  set  of  fields  displayed: 

g/keywd  ~  datai^base/  vol, issue 

where  the  regulair  expression  stands  for  the  set  of  all  strings  containing  the 
string  ‘data’  followed  by  0  or  more  arbitrary  characters  followed  by  the  string 
’base’. 

4.  Marking  and  Sorting 

Our  editor  provides  a  rhark  command  k  for  mahkitlg  Vith  a  tag  all  tiiples  of 
a  relation  that  satisfy  a  condition.  The  fact  of  being  marked  with  a  particular 
tag  becomes  a  property  of  a  tuple  and  can  be  used  in  a  predicate  as  part  of  a 
selection  criterion.  This  is  useful  when  we  want  to  restrict  our  attention  to  a 
subset  of  a  relation  or  when  we  want  to  break  a  complex  query  into  several 
steps.  For  example,  with  current  relation  P, 

g/vol=24/k  v24 
g/v24  Sc  $4=Fraser/ 

will  first  mark  with  tag  ’v24'  all  tuples  for  volume  24  and  then  display  a  subset 
of  them.  The  ’v24‘  tag  will  remain  associated  with  this  set  of  tuples  until 
another  mark  operation  with  the  saune  tag  is  performed,  or  the  tuples  are  expli¬ 
citly  untagged,  or  an  update  to  the  P  relation  is  performed.  The  idea  of  marking 
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a  set  of  tuples  was  independently  proposed  for  the  language  PLAIN  [vdR];  the 
concept  of  marking  they  describe  is  more  general  in  that  it  allows  a  subset  of 
the  columns  to  be- selected  for  marking,  while  we  only  mark  complete  tuples.  A 
similar  mechanism  is  implemented  by  the  hardware  of  the  P2AP  database 
machine  [OSS]. 

When  marking  a  set  of  tuples,  we  may  optionally  want  to  sort  it  at  the  same 
time: 

e  K 

g/keywd  ~  data/k  dat  [vol, Issue] 

will  mark  with  tag  'dat'  all  tuples  where  the  keyword  held  matches  'data'  and 
sort  them  in  increasing  order  of  volume  number  and  issue.  Decreasing  order 
can  be  obtained  by  using  \\  instead  of  [].  The  next  time  the  /dat/  command  is 
executed,  the  system  will  display  the  tuple  marked  ’daf  with  the  lowest  volume 
and  issue;  //  will  step  through  the  marking  in  order  of  volume  and  issue. 

*  «  r 

6.  Joins 

In  terms  of  the  relational  algebra,  we  have  described  so  far  how  to  perform 
selections  and  projections,  which  operate  within  the  current  relation.  In  our 
system,  joins  are  performed  by  comparing  attributes  from  different  relations. 
For  example,  if  the  current  relation  is  V,  then 

/$1=K.$1  &  dat/  $1,$2,K.$2,K.$3 

will  display  the  volume,  year,  issue,  and  page  number  of  all  papers  whose 
entries  in  the  K  relation  are  tagged  with  'dat'.  Our  system  restricts  tags  to  be 
unique,  so  it  can  tell  which  relation  the  tag  refers  to. 

The  rule  for  interpreting  comparisons  involving  attributes  from  different 
relations  is  the  following.  Every  field  reference  where  the  relation  name 
appears  explicitly  is  treated  as  an  existentially  quantified  variable.  All  field 
references  for  the  same  field  are  considered  the  same  variable  no  matter  in 
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which  form  they  appear.  This  rule  may  have  surprising  consequences;  for  exam¬ 
ple,  the  two  commands  below  are  different: 

/$1=$2/  - 

/$1=P.$2/ 

Assuming  P  is  the  current  relation,  the  first  one  selects  all  tuples  in  P  where 
the  first  and  second  fields  agree.  The  second  one  selects  all  the  tuples  such 
that  their  first  field  agrees  with  the  second  field  of  some  tuple  in  the  same  rela¬ 
tion. 

6.  Creating  New  delations 

Any  selection  predicate  can  be  followed  by  a  write  command  to  write  the 
output  of  the  selection  to  a  new  relation.  Thus,  with  current  relation  A, 

/$4  ~  Fraser  &  $1aP,$1  &  $2=P.$2  &  $3=Pi$3/w  $1  j$2;$3}Pi$4  PF 

will  create  a  new  relation  called  PF  in  the  database  and  store  in  it  the  volume. 

issue,  paige  number,  and  title  of  all  papers  written  by  Fraser.  Note  that  the 
names  and  data  types  of  the  columns  of  new  relations  are  inherited  from  the 
operand  relations.  We  have  not  addressed  the  problem  of  renaming  columns  if 
duplicate  neunes  occur. 


7.  Updating  Data 


There  are  four  corhriiarids  for  updating  relations:  insert,  substitute, 
change,  afid  delete.  Inserting  is  done  a  tuple  at  a  time  and  we  omit  lEe  details. 
The  substitute,  change  and  delete  commands  can  be  preceded  by  predicates 
indicating  where  they  are  to  be  applied;  otherwise,  the  current  tuple  is  con¬ 
sidered  to  be  the  target.  The  predicates  can  optionally  be  preceded  by  the  g 
prefix  to  mean  the  update  is  applied  to  ail  tuples  satisfying  the  predicate, 
rather  than  Just  the  next  one. 

The  substitute  command  is  used  to  change  one  or  more  fields  specified 


explicitly,  in  the  form 
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Ipredtlcate/s/ field  names'/neu;  values/ 

For  example,  if  the  current  relation  is  A, 

/ 

s/$3/298/ 

will  cheinge  the  page  number  of  the  current  tuple  to  296,  and 

/$1=20  &  $2=6/s/$3, $4/296, Zhou/ 

mil  find  the  ileit  tuj)la  whebe  Volume  is  20  and  iSsile  is  6  ahd  change  thfe  page 
number  to  296  and  the  author  to  Zhou.  The  commaind 


g/$4=Fraser/s/$4/fraser/ 

will  chainge  the  author  fbohi  Fraser  to  fraser  in  every  qualifying  tliple  of  the  A 
relation. 

Note  that  our  editor  does  not  allow  fine  intra-Meld  editing,  but  just  substi¬ 
tution  of  one  value  for  another.  It  would  be  useful  to  provide  the  user  with  all 
the  local  facilities  of  a  text  editor  on  fields  which  are  text  strings,  but  we  have 
not  incorporated  this  feature  to  keep  our  system  small. 


The  change  command  requires  typing  in  new  values  for  all  fields,  but  giv- 
the  field  name  or  number  as  the  new  value  for  a  field  will  leave  it 


unchanged-  For  example,  with  current  relation  V; 


g/vol=20/c/$1  ,$2,$3+1  / 

is  equivalent  to 

g/vol=20/s/$3/$3+1  / 

Both  will  add  1  to  the  page  number  for  all  tuples  in  the  V  relation  with  volume 
equal  to  20. 

The  delete  command  removes  the  next  (or  every  if  g  is  present)  tuple  satis¬ 
fying  the  predicate  from  the  current  relation.  For  example,  with  current  rela¬ 
tion  V, 


/vol<12/d 

will  delete  the  next  tuple  with  volume  less  than  12. 
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8.  Implementation 

A  prototype  version  of  the  editor  has  been  implemented  by  Sheriff  Ng  at 
CSRG.  It  is  written  in  C  using  yacc  and  lex  under  the  UNIX  operating  system.  The 
underlying  relational  database  system  is  Mistress  [MS],  currently  running 
under  several  versions  of  UNIX  on  PDP-ll’s  and  VAXes.  The  editor  uses  a  low  level 
tuple-at-a-time  interface  to  Mistress  to  retrieve  and  update  the  appropriate  sets 
of  tuples.  Markings  are  implemented  as  lists  of  tuple  identifiers. 


9.  CdftblUiWff 


We  have  sketched  a  design  for  an  interactive  query  language  for  relational 
databases.  Our  design  decisions  so  far  are  supported  only  by  introspection  and 
by  personal  experience  in  the  use  of  both  text  editors  and  different  types  of 
interactive  database  interfaces.  We  hope  to  use  our  prototype  as  a  tool  for  col¬ 
lecting  hard  evidence  and  feedback  on  the  usefulness  of  the  design. 


An  advantage  bf  ebfflbining  languages  and  text  editbfs  is  that  these 
are  tHe  two  user  interfaces  ori  wHlch  most  of  the  human  factors  experiments 
seem  to  have  coficehtfate^  [Re.EN].  Our  system  might  he  able  to  profit  from 
recommendations  arising  from  these  two  areas  of  study. 
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ABSTRACT 

One  of  the  major  activities  in  offices  is  communication.  Commun¬ 
ication  involves  information  exchange  in  different  kinds  of  media  such 
as  text,  voice,  pictures,  and  most  likely,  a  mix  of  these.  As  such,  a  suc¬ 
cessful  office  information  system  must  provide  adequate  facilities  for 
handling  these  kinds  of  communication  media.  This  paper  discusses 
the  design  and  implementation  of  a  voice  response  system  for  OFS 
(Office  Form  System),  a  prototype  office  information  system. 
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1.  INTRODUCTION 
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Office  automation  is  prompted  by  a  need  to  increase  the  productivity  of  office 
workers,  thereby  reducing  the  cost  of  operating  an  office.  It  involves  a  number  of  com¬ 
puter  science  disciplines  such  as  software  engineering,  database  management,  com¬ 
puter  architecture,  communication,  and  human  factors  [Ellis,  I960].  The  challenge  is 
to  apply  the  results  obtained  in  these  disciplines  to  an  office  information  system.  After 
some  years  of  research,  several  major  problem  areas  have  gradually  been  identified 
[Tsichritzis,  1962]  [Ellis,  1980]  [Tsichritzis  and  Lochovsky,  1980]. 

First,  facilities  for  automating  office  work  need  to  be  integrated.  Traditioncdly, 
such  facilities  include  database  management  systems,  electronic  mail,  word  proces¬ 
sors,  telephone,  facsimile,  typewriters,  photocopiers,  etc.  Unfortunately,  these  facili¬ 
ties  eire  often  provided  to  users  in  separate  pieces,  usually  supplied  by  different 
manufacturers  with  different  areas  of  expertise.  The  problem  confronting  office  infor¬ 
mation  systems  is  how  to  integrate  such  a  large  variety  of  facilities  into  a  single  system 
while  keeping  them  in  a  uniform  environment  with  a  clean  user  interface. 

Second,  office  procedures  must  be  automated,  rather  than  merely  mechanized. 
To  this  end,  users  must  be  able  to  specify  office  procedures  to  the  system  with  a  power¬ 
ful,  yet  easily  understandable,  specification  language.  Once  specified  to  the  system, 
appropriate  actions  can  be  initiated  by  the  system  without  the  intervention  of  humans 
when  certain  conditions  are  satisfied. 

Third,  tools  must  be  provided  to  represent  the  structure  of  the  office,  to  specify 
end  to  analyse  information  requirements  and  information  flow  within  an  office,  and  to 
predict  the  effects  of  changes  in  office  organization.  Such  tools  will  enable  managers 
to  restructure  the  office  systematically  to  suit  the  new  information  era.  In  addition, 
administrative  staff  can  use  these  tools  to  design  and  verify  office  procedures.  Exam¬ 
ples  of  office  modeling  tools  are  Office  Form  Flow  Model  [Ladd,  1980]  and  Information 
Control  Nets  [Ellis,  1980]. 
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As  a  subproblem  of  the  first  area  (L.e.,  integration  of  facilities),  the  integration  of 
communication  facilities  has  particular  importance.  Historically,  the  use  of  computers 
in  the  office  has  been  for  data  processing  (e.g.,  database  management  systems,  acount- 
ing,  and  payroll  preparation).  However,  the  major  activity  in  an  office  is  not  only  data 
processing  but,  with  equal  importance,  interpersonal  communication,  whether  formal 
or  informal.  Communication  involves  information  exchange  in  different  kinds  of  media 
such  as  text,  voice  and  pictures,  and  most  likely,  a  mix  of  these  media.  As  such,  a  suc¬ 
cessful  office  information  system  must  be  able  to  deal  with  both  foTTnatted  data  (e.g., 
data  represented  as  records)  and  laiformatted  data  (e.g.,  text,  voice  and  pictures). 

The  emphasis  of  this  paper  is  on  incorporating  speech  communication  in  an  office 
Information  system.  Specifically,  a  facility  for  synthesizing  speech  from  text  is  dis¬ 
cussed  and  its  design  and  implementation  within  OFS  (Office  Form  System  [Cheung, 
1980]),  a  prototype  office  information  system,  are  described. 

2.  OFS  (OFFICE  FORM  SYSTEM) 

OFS  is  a  prototype  office  Information  system  designed  and  developed  at  the 
University  of  Toronto.  A  brief  overview  of  its  purpose  and  function  cire  given  below.  A 
discussion  of  the  concept  of  form  management  can  be  found  in  [Tsichritzis,  1982]. 
Details  about  OFS  can  be  found  in  [Cheung,  1980]  [Tsichritzis,  1980]. 

A  form  can  be  used  as  a  concept  for  integrating  various  facilities  and  services  in 
an  office  information  system.  It  can  be  thought  of  as  a  logical  image  of  a  business 
paper  form  which  is  used  in  almost  every  office,  but  it  is  more  general,  flexible  and 
elaborate  than  an  ordinary  paper  form.  OFS  is  a  computerized  system  for  handling 
electronic  forms.  In  OFS,  forms  can  be  created,  retrieved,  copied,  printed,  mailed, 
traced  and  put  into  a  dossier  just  as  can  be  done  with  ordinary  business  paper  forms. 
However,  there  are  some  facilities  in  OFS  (existing  or  capable  of  being  incorporated  in 
the  future)  which  have  particular  importance  in  an  office  information  system. 
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.  / 

OFS  is  implemented  on  a  network  of  workstations  on  which  users  work.  As  such, 
forms  can  be  mailed  from  one  station  to  another  without  (observable)  delay.  Global 
querying  is  possible  in  that  information  about  forms  residing  in  other  workstations  can 
be  obtained.  OFS  is  compatible  with  a  database  management  system  (Micro  Relational 
System,  MRS  [Kornatowski,  1979]  [Hudyma,  1981])  so  that  information  about  forms  can 
be  queried  using  facilities  provided  by  the  database  management  system  and  by  OFS. 
In  the  former  case,  relational  commands  are  available  to  experienced  and  authorized 
users  to  operate  on  a  collection  of  forms.  This  allows  information  about  a  number  of 
forms  to  be  obtained  and  processed.  In  the  latter  case,  forms  are  processed  one  at  a 
time.  Forms  can  be  accessed  sequentisdly  or  by  specifying  conditions  .on  attribute 
values. 

In  the  general  case,  a  form  can  be  viewed  as  a  message  in  that  its  contents  convey 
information.  Information  exchange  is  actually  carried  out  when  a  form  is  mailed  from 
one  station  to  another.  Traditionally,  a  message  is  regarded  as  unformatted  and  it  is 
up  to  the  recipient  to  interpret  the  message.  In  an  electronic  mail  system,  a  message 
is  often  defined  as  a  header  and  a  body  [Kirstein,  1980].  However,  this  is  not  enough  to 
allow  flexible  message  processing  facilities,  such  as  mail  classification,  for  two  reasons 
[Tsichritzis,  1981].  First,  the  header  structure  is  uniform  for  all  messages,  so  it  can 
not  be  embedded  with  adequate  information  to  allow  the  system  to  interpret  a  large 
variety  of  messages.  Second,  the  header  structure  can  not  evolve  with  time. 

On  the  other  hand,  a  form  imposes  structure  on  a  message  while  maintaining  great 
flexibility  (i.e..  a  form  can  be  redesigned  and  a  user  can  add  as  many  form  types  to  the 
system  as  he  thinks  appropriate).  Furthermore,  the  structure  of  a  form  guarantees  a 
unique  interpretion  of  its  contents  which  would  be  otherwise  ambiguous,  if  not  mean¬ 
ingless.  For  instance,  the  attribute  value  ’’1980"  can  stand  for  a  year,  an  amount  or 
simply  a  serial  number.  But  if  this  string  appears  in  a  form  with  label  publication 
year",  then  it  stands  for  a  year;  the  message  conveyed  to  the  user  is  clearly  defined. 
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In  OFS,  attribute  values  are  extracted  from  a  form  and  stored  separately  as  a  rela¬ 
tion  tuple.  When  a  user  retrieves  a  form,  the  tuple  is  retrieved  and  filled  into  a  tem¬ 
plate  through  which  structure  and  meaning  are  assigned  to  the  attribute  values.  For 
instance,  a  text  template  can  be  defined  to  map  attribute  values  to  form  a  text  docu¬ 
ment.  Figure  1  shows  a  text  template  for  a  book  formi.  Attributes  are  numbered  from 
left  to  right,  top  to  bottom,  starting  with  1  for  the  form  key.  They  are  referenced  by 
their  numbers. 


FORM  KEY; _ 

BOOK  FORM 

CALL  NUMBER; _ 

TITLE; _ 

AUTHOR; _ 

PUBUCATION  YEAR: _  PRICE;, 

ENTERED  ON; _  STATION  ID:_ 


Figure.  1 

The  separation  of  attribute  values  and  templates  allows  flexibility  in  processing  a 
form.  For  one  thing,  the  attribute  values,  stored  internally  as  tuples,  can  be  accessed 
by  a  database  management  system  For  another,  the  same  set  of  attribute  values  can 
be  structured  in  different  ways  and  be  interpreted  differentl)^  by  the  system  by  using 
different  templates.  In  this  regard,  a  template  resembles  a  schema  and  subschema  in 
a  database  management  system. 
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Apart  from  defining  the  spatial  structure  of  a  form,  information  can  be  embedded 
into  a  template  so  that  the  system  knows  how  to  manipulate  the  attribute  values  as 
they  are  mapped  to  the  output  device.  Depending  on  the  template,  attributes  can  be 
mapped  onto  a  display  terminal,  a  speech  synthesizer,  a  graphics  terminal,  a  printer 
eind  so  on.  Hence,  templates  can  be  distinguished  according  to  their  function  as  text 
templates,  voice  templates,  graphic  templates,  printer  templates  and  the  like.  In  fact, 
a  mix  of  these  templates  can  be  used  if  a  form  uses  multi-media  output  (i.e,,  combining 
text,  pictures  and  voice  messages  in  one  form).  The  following  is  an  elaboration  of  some 
of  the  facilities  that  can  be  provided  by  templates 

Since  a  template  can  be  thought  of  as  a  general  mapping,  attribute  values  can  be 
modified  as  they  are  mapped  to  a  template.  An  attribute  can  be  attached  with  a  pro¬ 
cedure  which  takes  the  attribute  and,  optionally,  other  attributes  as  input  and  pro¬ 
duces  ein  output  to  the  template.  The  final  output  can  be  different  from  what  is  actu¬ 
ally  stored  in  the  system.  For  instance,  the  full  name  of  a  person  can  be  stored  in  the 
system  but  a  template  can  map  only  the  last  name,  the  last  name  with  imtials  or  map  a 
different  form  of  the  name  to  the  output.  Furthermore,  the  mapping  can  be  intelligent 
enough  to  determine  the  title  of  that  person  from  other  attribute  values  such  as  mari¬ 
tal  status,  sex  and  education  qualification. 

Since  all  attributes  are  represented  internally  as  a  bit  string,  it  is  necessary  to  tell 
the  system  how  to  interpret  the  bit  string.  Consider  a  graphics  template.  An  attribute 
cein  be  a  character  string,  in  which  case,  the  bits  are  interpreted  in  groups  of  eight. 
Further,  different  fonts  can  be  used  when  the  attributes  are  displayed.  On  the  other 
hand,  an  attribute  can  be  a  picture.  In  this  case,  the  cooi'dinates,  size  and  resolution  of 
the  picture  must  be  defined  to  the  system.  In  addition,  the  encoding  scheme  of  the  pic¬ 
ture  must  be  known.  It  can  be  a  bit  pattern  of  the  picture.  It  can  also  be  compressed 
with  a  certain  algorithm  or  simply  be  a  set  of  graphics  instructions  which  define  the 
picture.  All  this  information  can  be  defined  within  a  template  so  that  as  long  as  a  user 
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is  provided  with  a  suitable  template,  he  does  not  have  to  worry  about  the  internal 
representation  of  the  attributes.  This  has  an  implication  that  communication  between 
heterogeneous  systems  is  possible  if  the  template(s)  is  sent  along  with  its  attributes  to 
the  destination.  This  is  ver>’  important  in  an  office  information  system. 

Imposing  structure  on  messages  means  that  the  user  is  not  left  free  to  enter  any 
message  on  a  template.  He  must  enter  data  in  the  way  defined  by  the  structure  of  the 
template.  This  could  be  a  constraint  which  sacrifices  flexibility  but  it  is  a  way  to  avoid 
ambiguity  and  guarantee  consistent  interpretation  of  messages.  Furthermore,  the 
constraint  can  be  useful  in  some  situations.  For  instance,  if  a  field  is  an  utterance,  a 
voice  template  can  define  the  way  m  which  the  utterance  is  stored  (e.g.,  using  digitiza¬ 
tion,  LPC  or  formant  synthesis,  or  text-to-speech  synthesis  [Schafer,  1975]  [Maikhoui, 
1975],  [Flanagan.  1970],  [Allen,  1976])  and  the  rate  of  output.  In  text-to-speech  syn¬ 
thesis,  linguistic  ambiguities  would  arise.  Imposing  a  structure  on  the  text  can  help  to 
resolve  some  of  the  ambiguities. 

3.  VOICE  TEMPLATES 

In  a  voice  template,  as  in  a  text  template,  attributes  are  numbered  from  left  to 
right  and  from  top  to  bottom.  If  an  attribute  vadue  is  to  be  spoken  out.  it  must  have  a 
corresponding  entry  in  the  voice  template.  An  entry  consists  of  five  parts;  a  number,  a 
prefix  utterance,  a  trailing  utterance,  a  flag  and  a  dictionary.  Since  a  text-to-speech 
synthesizer  is  used  in  the  prototype  system,  attribute  values  which  are  to  be  spoken 
out.  the  prefix,  and  trailing  utterances  are  actually  stored  as  character  strings, 

If  an  entry  is  numbered  by  n,  it  is  associated  with  the  nth  attribute  value  in  a  rela¬ 
tion  tuple.  If  an  attribute,  n,  is  to  be  spoken  out,  an  entry,  numbered  n,  must  be  found 
in  the  voice  template. 

The  prefix  and  trailing  utterances  provide  a  similar  facility  as  "fill  in  the  slot"  in  a 
text  template  in  that  an  attribute  value  can  be  prefixed  or  appended  with  other  utter- 
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ances  as  it  is  spoken  out.  The  prefix  utterance  serves  to  qualify  the  attribute,  (e.g. 
"amount  is",  "customer  name  is",  etc.).  The  trailing  utterance  serves  primarily  as  the 
unit  of  the  attribute  value,  like  "dollar". 

The  dictionary  associated  with  each  attribute  serves  to  resolve  some  ling>uistic 
ambiguity  and  exceptions  not  handled  by  the  synthesizer  Words  separated  by  blanks, 
commas  or  a  period  are  stripped  out  of  the  attribute  value  and  are  matched  with  words 
in  the  dictionary.  If  a  match  is  found,  the  word  is  substituted  with  the  corresponding 
word  or  phrase  defined  in  the  dictionary.  Using  a  separate  dictionary  for  each  attri¬ 
bute  implies  that  substitution  of  a  word  can  be  done  according  to  the  context  in  wbuch 
It  appears.  This  helps  to  resolve  some  ambiguities  easily.  For  instance,  the  word  "mrs" 
can  be  the  abbreviation  for  "misses"  or  the  acronym  for  "Micro  Relational  System".  The 
form  administrator,  who  designs  the  voice  template,  can  determine  the  correct  con¬ 
text  and  specify  the  appropriate  substitution  in  the  dictionary  associated  ’v^^.th  the 
attribute  (e.g..  "misses"  should  be  used  if  "mrs"  appears  in  a  name  field  such  as  "custo¬ 
mer  name"  whereas  "};[RS"  should  be  used  if  "mrs"  appears  in  a  item  field  such  as  "pro¬ 
duct  name". 

The  problem  of  homographs  (i.e.,  the  same  word  such  as  the  word  "record"  having 
a  different  pronunciation  when  it  is  in  different  word  classes)  can  also  be  solved,  to  a 
certain  extent,  in  the  same  way.  If  a  field  of  a  form  is  so  strongly  "typed"  that  the  form 
administrator  can  be  sure  that  a  word  (e.g.  "record")  would  appear  in  this  field  only  as 
a  verb  (or  noun),  then  a  dictionary  entry  can  be  specified  to  make  the  appropriate  sub¬ 
stitution  to  reflect  the  stress  pattern.  Without  making  use  of  prior  knowledge  about 
the  characteristics  of  an  attribute  value,  these  linguistic  ambiguities  must  be  resolved, 
by  performing  complex  syntax  and  semantic  analysis  to  determine  the  word  class  o: 
each  word  [Cherry,  1978].  Of  course,  there  are  some  situations  in  which  little  informa¬ 
tion  can  be  known  beforehand.  For  instance,  a  comment  field  would  be  so  uncon¬ 
strained  that  no  prior  know! edge  about  its  contents  can  be  derived  to  resolve  any 
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ambiguity.  If  correct  speech  output  is  desired,  a  procediure  which  can  emalyse  the  syn¬ 
tax  and  semantics  of  natural  language  should  be  associated  with  this  field.  In  any  case, 
a  great  saving  in  processing  time  can  still  be  achieved  since  speech  output  of  this  kind 
of  field  would  not  be  frequent. 

In  some  situations,  brute  force  substitution  is  not  effective,  if  at  all  possible.  For 
fields  whose  input  stands  for  an  amount,  the  string  of  numbers  should  first  be  con¬ 
verted  to  speech  forms,  (e.g.,  ”123"  should  be  converted  to  "one  hundred  twenty 
three”).  Another  example  is  the  "date”  field  in  which  "sat"  should  be  converted  to 
"Saturday”.  This  kind  of  conversion  is  done  by  specifying  a  conversion  procedure  for 
the  field  that  needs  conversion.  As  such,  a  ftag  field  is  provided  to  qualify  the  attribute 
value  so  as  to  invoke  the  appropriate  procedure.  Examples  of  flags  cire; 

Y  (for  year)  which  causes  a  number  to  be  spoken  as  a  year  in  the  conventional 

way.  For  instance,  "2000”  is  translated  to  "two  thousand",  "1960"  is 
translated  to  "nineteen  eighty". 

D  (for  date)  which  expands  a  date  format  to  its  full  text.  For  instance,  "Thu  Oct 

1  1980"  is  translated  into  "Thursday  October  first  nineteen  eighty”. 

A  (for  cimount)  which  causes  a  number  to  be  spoken  out  as  a  quantity.  For  instance, 

"1960"  is  spoken  as  "one  thousand  nine  hundred  and  eighty". 

If  no  flag  is  specified,  a  number  will  be  read  out  digit  by  digit.  Other  procedures 
can  also  be  attached  to  an  attribute  by  specifying  other  flags  if  the  form  administrator 
feels  that  it  is  necessary. 

Utterances  can  be  defined  in  a  voice  template  such  that  they  are  spoken  before 
any  attribute  is  spoken  or  after  all  attributes  are  spoken.  These  can  be  used,  for 
instance,  for  informing  users  of  the  form  type  of  the  form  being  processed  or  for 
greeting  or  alerting  messages. 

In  a  voice  template,  an  entry  which  corresponds  to  an  attribute  value  has  the  fol- 
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lomng  general  format; 


attribute-number  prefix_utter once, trailing-utterance, flag 
dictionary-entry  :  substitution 


Figure  2  is  a  voice  template  defined  for  the  book  form  given  in  figure  1. 


“2  book  form 

2  call  number 

3  title  is 

MRS  :  M  R  S 

OFS  ;  office  form  system 

4  authored  by 

5  published  in..Y 

6  price  is. dollars. A 

7  entered  on,.D 
-1  form  end 


Figure.  2 

The  sentence  labelled  "-2"  •will  be  spoken  before  any  other  utterance.  The  sen¬ 
tence  "-1"  will  be  spoken  after  all  attributes  have  been  output.  Negative  values  are  used 
to  distinguish  them  from  other  entries  which  correspond  to  attribute  values  of  the 
form.  Since  no  entry  is  labelled  by  "I”  or  ”8”,  the  form  key  and  the  station  id  will  not 
be  spoken  out.  If  no  voice  template  is  defined  for  a  form  type,  then  no  voice  output  will 
result  when  a  form  of  that  type  is  displayed.  The  rest  of  the  template  is  defined  as 
described  above.  Like  text  templates,  voice  templates  can  be  created  and  modified 
with  an  editor.  They  are  managed  by  the  form  administrator  Ordinary  users  are  not 
allowed  to  modify  them. 
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4.  IMPI.EMENTAnON  ISSUES 

OFS  is  •written  in  the  programming  language  C  [Kernighan,  1978]  and  runs  under 
the  UNIX  operating  system  [Thompson,  1976]  on  a  LSI  11/23.  The  voice  response  sys¬ 
tem  is  also  implemented  in  the  same  environment. 

The  synthesizer  used  in  the  system  is  a  text-to-speech  synthesizer  from  Votreix 
[Votrax,  1980].  It  has  a  RS-232  serial  interface  and  accepts  characters  as  input.  As 
such,  it  Ccin  be  treated  by  the  computer  as  a  terminal.  In  the  current  implementation, 
a  TTY  driver  is  used  for  interfacing  the  synthesizer.  The  synthesizer  also  has  an  inter¬ 
nal  buffer  for  holding  input  characters.  Two  modes  of  operation  are  available  First,  it 
interprets  the  characters  in  the  buffer  as  English  text  and  uses  a  text-to-speech  algo¬ 
rithm  to  translate  the  text  into  speech  directly.  Second,  it  interprets  the  characters 
as  phonemes  and  as  such  operates  as  a  phonetics  synthesizer.  It  is  possible  to  switch 
between  these  two  modes  under  software  control  by  sending  a  sequence  of  control 
characters. 

The  text-to-speech  algorithm  can  translate  85%  of  the  words  correctly.  To  handle 
the  exceptions,  such  as  words  adapted  from  foreign  languages,  proper  nouns,  etc., 
another  dictionary  is  used.  Every  word  must  be  matched  with  this  dictionary  before  it 
is  output  to  the  s3Tithesizer.  If  a  match  is  found,  the  corresponding  substitution  is 
made.  To  distinguish  between  this  dictionary  and  the  dictioneiries  defined  in  a  voice 
template,  it  is  called  the  global  dictionary  while  dictionaries  defined  in  a  voice  tem¬ 
plate  are  called  local  dictionaries.  They  are  different  from  each  other.  First,  a  local 
dictionary  applies  to  a  single  field  of  the  corresponding  form  while  the  global  dictionary 
applies  to  all  words  which  are  to  be  output  to  the  synthesizer.  Second,  in  local  dic¬ 
tionaries,  the  substitutions  are  specified  as  English  text  (i.e.,  they  are  translated  by 
the  same  text-to-speech  algorithm).  In  the  global  dictionary,  phonemes  are  used  in  the 
substitution.  The  reason  for  using  English  text  in  a  voice  template  is  to  make  the  voice 
templates  independent  of  the  synthesizer  used  since  the  phonemes  used  by  different 
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synthesizers  cam  be  different.  The  form  administrator  can  assume  that  a  perfect  syn¬ 
thesizer  is  available  to  him  w^hen  he  defines  voice  templates.  The  global  dictionary  is 
used  to  supplement  the  synthesizer  so  that  it  looks  like  a  perfect  one. 

Options  are  provided  in  OFS  so  that  a  user  can  enable  or  disable  the  voice  output 
at  any  time  during  a  work  session.  This  is  important  since  the  usefulness  of  voice  out¬ 
put  depends  on  the  application.  For  instance,  an  experienced  user  may  find  the  voice 
output  disturbing  while  a  novice  user  or  a  user  who  is  too  busy  to  look  at  the  screen 
may  find  the  voice  output  very  useful  An  option  is  also  provided  to  users  so  that  the 
output  of  the  current  form  to  the  synthesizer  can  be  repeated. 

When  voice  output  is  enabled,  an  attribute  value  which  has  a  corresponding  entry 
in  the  voice  template  is  processed  as  follows: 

1)  The  flag  of  the  attribute  is  checked.  If  a  flag  is  found,  the  attribute  value  is  pro¬ 
cessed  by  the  procedure  specified  by  the  flag  and  steps  (4)  and  (5)  are  executed 
next. 

2)  If  a  flag  is  not  found,  the  words  in  the  attribute  value  are  stripped  out  and 
matched  to  its  local  dictionary.  If  a  match  is  found,  the  word  is  substituted 
accordingly. 

3)  The  processed  attribute  value  is  then  concatenated  with  the  prefix  and  trailing 
utterances  defined  in  the  voice  template  to  form  a  new  string. 

4)  The  words  in  the  resulting  string  are  stripped  out  again  and  matched  to  the  global 
dictionary.  If  a  match  is  found,  a  word  is  substituted  by  the  corresponding 
phonemes  together  with  control  characters  which  switch  the  synthesizer  from  text 
to  phoneme  mode  and  switch  it  back  to  text  mode  aiter  the  phonemes  are  output. 

5)  The  processed  string  is  output  to  the  S3mthesizer. 

From  usage,  it  has  been  found  that  the  speech  output  for  this  system  is  quite 
intelligible.  Since  the  attribute  values  of  a  form  are  usually  very  short,  the  intonation 
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is  not  very  important. 

5.  CONCLUSION 

Speech  ;ofTers  humans  a  means  of  spontaneous  and  effective  communication. 
Information  exchange  can  be  effected  as  soon  as  a  message  is  formulated  in  the  brain; 
no  tool,  no  preparation  is  required.  In  en  office,  speech  is  even  more  important  for 
communication  between  people.  As  such,  an  office  information  system  must  provide 
adequate  facilities  for  handling  speech.  Those  facilities  include  voice  editing,  voice 
filing,  voice  electronic  mail,  and  voice  synthesis  and  recognition  [Maxemchuk,  1980], 
[Nacon,  1980],  [Koekebakker,  1981].  An  important  requirement  is  the  capability  of 
handling  voice  and  other  media  such  as  text  and  pictures  in  an  integrated  v,ray.  These 
facilities  are  important  in  reducing  communication  cost  and  increasing  human  accep¬ 
tance  of  an  office  information  system.  The  help  of  a  voice  response  system  to  blind 
people  is  obvious. 

In  an  office  environment,  no  fixed  pattern  of  verbal  communication  can  be  cap¬ 
tured  a  priori.  Thus,  the  system  must  be  able  to  synthesize  speech  from  unrestricted 
text.  As  such,  a  text-to-speech  synthesizer  is  used  in  our  prototype  system.  The  voice 
response  system  enables  any  field  of  a  form  to  be  spoken  out  in  a  meaningful  way  by 
using  a  voice  template.  An  attribute  is  qualified  with  a  prefix  utterance  such  as 
"amount  is"  and  followed  by  another  one  such  as  "dollars"  when  it  is  output  from  the 
synthesizer.  Furthermore,  the  way  to  speak  an  attribute  can  be  modified  according  to 
context.  For  instance,  "1978"  is  translated  into  "nineteen  seventy  eight"  when 
representing  a  year  and  "one  thousand  nine  hundred  and  seventy  eight"  when 
representing  an  amount.  The  system  also  enables  voice  messages,  as  opposed  to  attri¬ 
butes.  to  be  embedded  in  a  form.  The  linguistic  ambiguities  inherent  in  text-to-speech 
synthesis  are  handled  by  associating  each  field  with  a  dictionary  which  is  defined  in  the 
voice  template. 
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Text-to-speech  synthesis  is  the  most  economic  way  for  implementing  an  unlimited 
vocabulary  voice  response  system.  The  most  serious  drawback  is  its  low  speech  qual¬ 
ity.  However,  research  effort  has  been  directed  to  developing  rules  to  translate  text  to 
speech  with  prosodic  features  (e.g.,  stress  patterns)  [Smith.  1981].  With  the  low  cost  of 
memory  and  high  processing  power  of  microprocessors,  such  a  system  can  soon  be 
expected  to  be  implemented  on  a  microprocessor. 
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/iistract  Office  information  models  are  used  to  represent  the  operations  and  in¬ 
formation  structure  of  Office  Information  Systems.  This  paper  determines  re¬ 
quirements  for  these  models  by  examining  the  functionality  of  Office  Informa¬ 
tion  Systems.  The  paper  then  discusses  the  representation  of  'office  objects’ 
and  their  associated  operations  of  filing,  mailing  and  formatting.  This  represen¬ 
tation  is  based  upon  concepts  from  the  field  of  semantic  data  modelling. 


1.  Introduction 

Computer-based  Office  Information  Systems  (OIS’s)  are  concerned  mth  the 
representation  euid  management  of  information  relevant  to  offices.  Here  'information' 
is  used  in  the  sense  of  interpreted  data;  the  OIS  not  only  stores  raw  data  but  also  pro¬ 
vides  facilities  for  the  interpretation  of  this  data  by  the  end  user. 

The  logical  structure  of  data  is  specified  by  means  of  a  data  model.  Data 
models  also  support  primitive  operations  for  manipulating  data.  For  example,  the 
three  tradition^  data  models,  the  network,  hierarchical  and  relational  models,  organ¬ 
ize  data  using  networks,  trees  and  tables  respectively.  Primitive  operations  include 
such  things  as  updating  and  retrieving  units  of  data.  In  order  to  describe  the  informa¬ 
tion  content  of  a  system  we  will  make  use  of  what  may  be  termed  an  information 
model.  Such  a  model,  in  addition  to  specifying  data  structure  and  primitive  opera¬ 
tions.  contains  constructs  which  aid  in  the  interpretation  of  data. 

An  office  information  model  is  an  information  model  designed  specifically  for 
the  office  domain.  The  second  section  of  this  paper  discusses  the  role  of  these  models 
in  relation  to  the  OIS  and  identifies  a  number  of  requirements  for  such  models.  The 
third  section  shows  how  various  concepts  from  the  field  of  semantic  data  modelling 
may  be  extended,  providing  a  basis  for  the  representation  of  'office  objects’  by  office 
information  models. 


2.  Office  Information  Models 


Two  of  the  major  problems  facing  the  development  of  Office  Information  Sys¬ 
tems  are  integration  and  the  design  of  suitable  user  interfaces  [5,  16],  Both  problems 
stem  from  the  diversity  of  OIS  functionality  and  the  associated  tendency  to  divide  the 
OIS  into  numerous  semi-independent  subsystems.  This  approach  discourages  integra¬ 
tion  since  each  subsystem,  such  as  electronic  mail,  information  retrieval,  document 
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preparation  etc.,  may  have  its  own  representational  peculiarities  which  complicate  the 
exchange  of  information  between  subsystems.  In  addition  the  user  interface  may 
become  complex  as  each  subsystem  will  phrase  its  operations  in  a  possibly  different 
command  lajiguage. 

Office  information  models  address  the  problems  of  integration  and  interface 
design  by  providing  a  single  data  representation  and  a  uniform  set  of  operations.  The 
particular  model  described  in  this  paper  supports  the  representation  of  'office  objects'. 
These  are  the  mlormation  bearing  objects  found  in  offices;  examples  are  forms,  letters, 
dossiers  and  so  on.  The  use  of  interpretative  information  within  the  model  allows  the 
description  of  an  object’s  structure  at  two  levels.  The  first  is  data  oriented,  suitable  for 
the  specification  of  operations.  The  second  level  is  information  oriented  and  intended 
for  user  Interaction.  We  will  borrow  the  terms  datalogical  and  infological  [lO]  to  distin¬ 
guish  these  two  levels. 

Before  introducing  the  model  let  us  consider  some  of  the  requirements  of 
office  information  models  in  general.  We  may  divide  these  requirements  into  those  for 
the  datalogical  level  and  those  for  the  infological  level.  At  the  datalogical  level  we  will 
require  that: 

•  The  model  must  support  the  different  types  of  data  found  within  the  office. 

•  The  model  must  support  some  aspect  of  communication 

These  two  requirements  arise  from  the  OIS  functions  of  information  management 
(storage,  acquisition  and  retrieval)  and  communication  [16].  Although  Office  Informa¬ 
tion  Systems  may  provide  a  number  of  other  functions,  such  as  decision  support  or 
calendar  management,  the  information  management  and  communication  functions  are 
in  some  sense  minimal;  it  is  difficult  to  classify  as  an  OIS  a  system  which  does  not  sup¬ 
port  these  functions. 

The  first  of  the  above  requirements  refers  to  the  various  types  of  data  used 
within  the  office.  These  include  strictly  formatted  numeric  and  character  data  such  as 
found  in  tables  or  forms,  text  and  graphic  data  such  as  found  in  documents,  and  audio 
data  such  as  may  be  found  in  messages.  Most  current  data  models  treat  only  strictly 
formatted  data.  However,  the  use  of  non-strictly  formatted  data  is  widespread  within 
the  office.  Any  system  which  attempts  to  support  office  activities  must  talce  this  into 
account. 


The  second  requirement  derives  from  the  distributed  configuration  of  Office 
Information  Systems.  One  may  view  an  OIS  as  a  network  of  information  bearing  sta¬ 
tions.  A  very  common  operation  within  the  OIS  is  the  transfer  of  information  from  one 
station  to  another.  An  office  information  model  should  support  the  station  concept  and 
allow  information  transfer  between  stations. 

Requirements  for  the  infological  level  are  related  to  the  humein  factors  issues 
of  data  model  design  [12].  Specifically,  we  will  require  that: 

•  The  model  should  make  use  of  natural  structure. 

•  The  model  should  be  easily  extended. 

While  the  above  two  requirements  are  desirable  in  most  computer  systems,  they  merit 
repeating  since  ease  of  use  and  flexibility  are  crucial  to  the  acceptance  of  Office  Infor¬ 
mation  Systems. 

The  requirement  for  natural  structure,  i.e.,  that  perceived  by  the  user,  allows 
the  data  representation  used  by  the  office  information  model  to  be  based  upon  con¬ 
cepts  familiar  to  the  user.  This  aids  the  user  in  interpreting  data  and  performing 
operations  within  the  OIS. 
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The  extensibility  requirement  is  particularly  important  for  Office  Information 
Systems,  The  office  is  a  dynamic  environment  in  which  procedural  and  structural 
changes  are  to  be  expected.  The  OIS  must  be  easily  extended  to  reflect  these  changes 
as  they  occur. 

The  field  of  sememtic  data  modelling  [3,  3,  8,  15.  17,  20,  21]  has  long  been  con¬ 
cerned  with  the  questions  of  natural  structure  and  extensibility.  Semantic  data  models 
thus  provide  a  good  starting  :point  for  the  design  of  an  office  information  model,  How-_ 
ever,  these  models  lack  the  data  types  and  communication  mechanism  required  by 
Office  Information  Systems,  The  following  section  discusses  how  concepts  from  seman¬ 
tic  data  modelling  may  be  extended  to  the  office  domain,  A  rudimentary  office  infor¬ 
mation  model,  and  its  representation  of  filing,  mailing  and  formatting,  is  described. 


3.  Office  Object  Representation 

Semantic  data  models  often  meike  use  of  abstract  objects  which  correspond 
directly  to  objects  in  the  real  world.  The  analogous  concept  in  office  information 
models  is  that  of  ojfice  object.  An  office  object  is  any  information  found  in  the  office 
which  we  wish  to  be  individually  represented  within  the  model.  For  example,  we  could 
choose  to  represent  documents  as  office  objects.  In  this  case  such  things  as  sections 
of  documents  or  the  pages  of  a  document  cannot  themselves  become  office  objects. 
This  does  not  imply  that  we  cannot  represent  the  internal  structure  of  documents, 
merely  that  documents  are  to  be  taken  as  indivisible  or  primary  objects. 

An  object*  is  described  by  its  properties  and  their  values.  A  property  may  be 
single  vaLued  or  rmiltivalued,  in  the  latter  case  the  value  of  the  property  for  a  given 
object  may  be  a  set.  For  instance,  an  object  such  as  an  application  form  may  have  a 
property  used  to  record  the  telephone  number  of  the  applicant.  If  the  applicant  is  only 
allowed  to  give  one  telephone  number  then  the  property  is  single  valued.  If  the  appli¬ 
cant  may  give  more  than  one  telephone  number,  the  property  must  be  multivalued. 

Properties  are  either  composite  or  primitive,  A  primitive  property  heis  ein 
underlying  data  t3q>e.  The  allowable  data  types  should  include  boolean,  numeric,  char¬ 
acter,  graphic  and  audio.  The  first  three  data  types  are  those  most  commonly  found  in 
current  data  models.  Boolean  and  numeric  properties  are  useful  for  describing  strictly 
formatted  objects  such  as  forms.  Character  properties  may  also  be  used  with  forms 
and  any  object  which  contains  text.  The  graphic  data  type  is  used  for  digitized  images, 
the  audio  data  type  for  digitized  voice.  It  is  assumed  that  there  is  no  selective 
retrieval  based  upon  audio  or  graphic  property  values.  However,  graphic  or  audio 
information  may  be  retrieved  by  using  associated  character,  numeric  or  boolean  infor¬ 
mation.  This  is  similar  to  the  use  of  keywords  as  a  basis  for  text  retrieval. 

Composite  properties  are  formed  from  a  number  of  sub-properties  which  may 
in  turn  be  either  composite  or  primitive.  The  value  of  a  composite  property  is  the 
cross  product  of  the  values  of  its  sub-properties.  For  example,  the  composite  property 
address  could  be  formed  from  the  primitive  sub-properties  fxx>vince,  city,  street  and 
street-no.  A  typical  value  for  this  property  could  be  (Ontario,  Toronto,  Beverley,  210). 

Collection  of  objects  with  similar  properties  are  known  as  object  types.  Each 
member  of  an  object  type  is  said  to  be  an  instance  of  the  object  type.  An  object  type  is 
defined  by  specifying  the  properties  of  its  instances.  For  example,  the  object  type 


•  Office  objects  will  simply  be  referred  to  as  objects  when  the  meaning  is  clear  from  context. 
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order-form  may  be  defined  as: 

order^orm  —>  date  item+  total 

item  —>  quantity  part-no  price 

date  ~>  character 

total  — >  numeric 

quantity  ->  numeric 

part-no  — >  numeric 

price  —>  numeric 

An  order-form  has  three  properties:  date,  item  and  total.  The  properties  date  and 
total  are  primitive,  usin^  the  character  and  numeric  data  types.  The  item  property  is 
composite,  consisting  of  the  three  primitive  properties  quantity,  part-no  and  price, 
each  of  which  is  numeric.  In  addition  the  item  property  is  multivalued,  this  is  indi¬ 
cated  by  the  in  the  object  type  definition.  An  instance  of  the  object  type  order- 
form  is  shown  in  Figure  1. 

The  specification  oL  a  property’s  data  type  Imposes  a  constraint  on  the  physi¬ 
cal  representation  of  the  property's  values.  The  exact  physical  representations  of  the 
various  data  types  need  not  concern  us  here.  It  is  assumed  that  once  the  system 
knows  the  data  type  of  a  property  value  it  can  process  the  value  accordingly. 

Semantic  constraints  on  property  values  are  also  necessary.  For  example, 
the  part-no  :property  of  order^orm  must  only  be  assigned  valid  part  numbers.  Also 
certain  property  values  may  be  derived  from  the  values  of  other  properties.  Here 
order^orm  again  provides  an  example,  since  total  should  equal  quantity  times  price. 
However,  a  discussion  of  the  specification  of  semantic  constraints  and  derived  proper¬ 
ties  is  outside  the  scope  of  this  paper. 

As  a  second  example  of  object  type  definition  consider: 

letter  — >  sender  receptant  bod^  date 

bocfy  — >  paragraph4- 

paragraph  — >  sentence-i- 

sender  -->  character 

receptant  ->  character 

date  ~>  character 

sentence  — >  character 

The  letter  object  t3q>e  is  a  simple  text  document.  In  general  text  documents  are 
hierarchically  structured.  Here  letter  contains  a  body  with  a  variable  number  of  para¬ 
graphs,  a  paragraph  may  be  further  decomposed  into  a  variable  number  of  sentences 
(see  Figure  2).  Hierarchically  structured  documents  have  recently  been  used  in  a 
number  of  text  composition  systems  [l,  7,  19].  Document  structure  may  be  used  to 
specify  editing  operations  [7].  For  example,  the  operation  which  removes  a  sentence 
from  a  document  can  be  easily  visualized  as  a  reorganization  of  the  underl3dng  hierar¬ 
chy.  In  addition  document  structure  is  useful  during  retrieval  [22].  Thus,  the  user 
may  ask  to  see  the  structural  skeleton  of  a  document  or  to  view  a  particular  structural 
component. 

A  final  example  of  an  object  type  definition  is: 

employee-record  -->  sin  name  birthdate  salaiy  photo 
sin  — >  numeric 


order-form 


Figure  1  An  order-form  Instance. 


Figure  2  A  letter  instance  (property  values  not  shown). 
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name  — >  character 
birt-hdate  -->  character 
sadEiry  — >  numeric 
photo  ">  graphic 

The  employee-record  object  type  contains  a  graphic  valued  property,  here  used  for  an 
employee’s  photograph.  An  instance  of  the  employee-record  object  type  is  shown  in 
Figure  3.  Queries  with  selection  conditions  on  photo  values  are  not  allowed.  However, 
it  would  be  possible  to  retrieve  particular  photo  values  by  specifying  associated  infor¬ 
mation  such  as  ain  (social  insurance  number)  or  name  values. 

Figures  1  to  3  illustrate  the  hierarchical  structure  of  the  properties  of  office 
objects.  Hierarchical  organization  of  properties  is  found  in  a  number  of  semantic  data 
models  [3,  30,  21]  and  has  also  been  used  in  office  oriented  data  models  [13,  14].  An 
advantage  of  this  organization  is  that  it  lets  us  view  the  properties  of  objects  at 
different  levels  of  detail.  Furthermore,  similar  organization  may  be  found  in  natural 
language,  where  objects  (nouns)  may  be  described  by  property  hierarchies  [21]. 

Different  object  types  within  the  office  may  be  related  by  the  is-a  relationship 
[17].  Given  object  types  X  and  Y,  we  say  X  is-a  Y  if  every  instance  of  X  is  also  an 
instance  of  Y.  The  object  type  X  is  said  to  be  a  specialization  of  Y  (conversely  Y  is  a 
generalization  of  X)  and  may  contain  additional  properties  to  those  of  Y.  For  example, 
a  speciEilization  of  letter  could  be  letter-of-reference.  Specialization  and  generaliza¬ 
tion  are  commonly  used  in  semantic  data  models  [15]  and  also  object  oriented  systems 
such  as  Smalltalk  [6]  and  SBA  [4].  An  advantage  of  using  is-a  is  that  the  system  may  be 
easily  extended  by  specializing  previously  defined  types. 

The  various  is-a  relationships  that  exist  between  object  types  may  be  dep¬ 
icted  by  a  graph  with  object  types  as  nodes  and  arcs  corresponding  to  is-a  relation¬ 
ships.  Although  in  general  this  graph  may  be  a  network,  it  is  often  referred  to  as  an  is- 
a  hierarchy.  The  is-a  hierarchy  for  office  objects  is  shown  in  Figure  4  (here  the  direc¬ 
tion  of  increasing  specialization  is  downwards).  The  most  general  type,  office-object, 
has  all  office  objects  as  its  instances. 

Each  object  type  may  be  further  characterized  as  simple  or  aggregate. 
Aggregate  objects  may  ’contain’  other  objects.  Containment  is  expressed  using  the 
part-of  relationship.  For  example,  given  the  aggregate  object  x  (i.e..  x  is  an  instance  of 
an  aggregate  object  type)  and  some  object  y,  then  we  say  y  partof  z  if  x  contains  y. 
Simple  objects  are  those  not  allowed  to  appear  on  the  right  hand  side  of  the  part-of 
relationship.  However,  simple  objects  can  be  contained  within  aggregate  objects.  This 
method  of  aggregation  is  similar  to  the  'cover-aggregation'  of  RM/T  [3].  Here  though, 
part-of,  rather  than  type  membership,  is  used  to  relate  the  constituents  of  an  aggre¬ 
gate  instamce.  Consequently  aggregate  t3^es  are  not  elevated  to  the  level  of  metac¬ 
lasses  and  can  be  directly  included  within  the  is-a  hierarchy. 

A  natural  operation  on  an  aggregate  object  is  to  remove  a  constituent  and 
include  the  constituent  within  a  second  aggregate.  When  this  occurs  we  will  say  that 
the  first  object  is  'put'  into  the  second  object  or  that  the  first  object  has  'moved'. 
Depending  upon  the  aggregate  objects  used,  this  operation  may  represent  such  diverse 
user  activities  as  filing,  mailing  and  formatting.  In  the  following,  common  aggregate 
object  types  are  described. 
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3. 1  PHes  and  Dossiers 

The  definition  of  an  aggregate  object  type  can  include  constraints  on  the 
allowable  constituents  of  the  aggregate's  instances.  A  jllQ  is  an  ciggregate  object  type 
for  which  the  constituents  of  all  instances  belong  to  a  single  type.  For  example,  we  can 
define  an  aggregate  object  type  named  order-file  and  specify  that  the  constituents  of 
instances  of  order-file  be  of  type  ordeHlorm.  If  y  is  an  instance  of  order-file  and  x 
part-of  y  holds,  then  x  must  be  an  instance  of  order-form.  Alternatively,  we  may  define  ., 
an  aggregate  object  type  with  no  constraints  on  constituents.  Such  objects  are  called 
dossiers.  For  example,  let  d  be  an  instance  of  the  aggregate  type  personnel-dossier,  x 
an  instance  of  the  object  tjrpe  job-application,  and  y  an  instance  of  the  object  type 
resume.  It  is  not  necessary  for  x  and  y  to  be  of  the  same  type  if  they  are  both  consti¬ 
tuents  of  d,  i.e.,  both  x  part-of  d  end  y  part-of  d  are  possible.  The  part-of  relationship 
edlows  one  aggregate  object  to  be  part-of  another.  So,  if  dl  and  dB  are  both  dossier 
instances  then  dl  part-of  dS  is  also  possible. 

Properties  of  an  aggregate  object  can  be  used  for  descriptive  or  summary 
information  about  its  constituents.  Thus  a  property  of  ordernfile  could  be  number-of- 
orders,  where  for  each  instance  /  of  order-file  the  value  of  number-of -orders  is  equal  to 
the  number  of  constituents  of  /.  It  is  not  necessary  for  the  property  values  of  an 
aggregate  object  to  uniquely  identify  the  object  since  the  object’s  constituents  can  be 
used  to  diflerentiate  between  instances. 

The  filing  of  an  object  is  represented  by  including  the  object  as  a  constituent 
of  a  file  or  dossier  instance.  Two  methods  of  retrieval  are  possible.  First,  by  viewing 
the  object  as  an  instance  of  a  type  and  specifying  a  selection  condition  involving  the 
type's  properties.  Second,  by  viewing  the  object  as  a  constituent  of  an  aggregate.  If 
the  aggregate  is  homogeneous,  as  in  the  case  of  files,  then  again  selection  conditions 
based  on  property  values  may  be  given.  For  nonhomogeneous  aggregates,  i.e.,  dos¬ 
siers,  more  general  conditions,  such  as  pattern  matching,  may  be  specified. 


3,2  Stations 

The  part-of  relationship  imposes  a  hierarchy  on  object  instances.  The  root  of 
a  par f-o/ hierarchy  is  always  an  instance  of  the  station  aggregate  object  type.  The  pur¬ 
pose  of  station  is  to  group  together  all  objects  which  belong  to  a  single  station  within 
the  OIS.  In  a  distributed  configuration  it  will  be  assumed  that  all  constituent  objects  of 
a  station  instance  are  located  on  the  same  node.  Transfer  of  an  object  from  one  sta¬ 
tion  to  another  is  represented  by  removing  the  object  from  the  parf-o/ hierarchy  of  the 
first  station  and  placing  it  under  that  of  the  second  station.  Rather  than  having  the 
user  move  the  object  directly  to  the  second  station  it  is  convenient  to  introduce  the 
in-tray  and  out-tray  aggregate  object  types.  Typically  a  station  contains  both  an  in- 
tray  and  an  out-tray  instance.  When  an  object  is  put  into  an  out-tray  the  OIS  automati¬ 
cally  transfers  the  object  to  the  in-tray  of  the  destination  station. 

There  is  a  problem,  though,  in  associating  routing  information  with  the  consti¬ 
tuents  of  in-trays  and  out-trays.  Consider  the  in-tray  as  an  example,  somehow,  in 
order  to  perform  the  transfer,  the  OIS  must  know  the  destination  of  each  constituent. 
Assigning  in-tray  a  multivalued  property  for  use  as  a  destination  list  is  not  possible 
since  there  is  no  way  to  match  up  the  property's  values  with  constituents.  An  alterna¬ 
tive  is  to  allow  each  in-tray  only  a  single  constituent  at  a  time,  in  this  case  in-tray 
could  have  a  single  valued  property  used  to  record  the  destination.  However,  this 
could  frustrate  the  user,  particularly  if  there  are  delays  in  object  transfer.  A  second 
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Figure  4  The  is-a  object  type  hier^'chy. 
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alternative  is  to  require  that  each  object  type  have  special  properties  which  are 
reserved  for  routing  information.  This  is  unsatisfactory  though,  since  changes  in  the 
mailing  system  potentially  affect  all  previously  defined  object  types.  A  more  realistic 
solution  makes  use  of  the  envelope  aggregate  object  type.  Among  the  properties  of 
envelope  are  to  and  from  which  are  used  to  identify  the  destination  and  source  sta¬ 
tions.  Additional  properties  could  be  used  to  specify  such  things  as  service  class,  if  the 
OIS  provides  different  classes  of  service,  or  perhaps  security  information. 

Suppose  the  user  of  station  si  wishes  to  transfer  the  object  x  to  station  sS. 
(It  is  not  necessary  for  x  to  be  a  simple  object,  if  z  is  an  aggregate  then  the  consti¬ 
tuents  of  X  are  transferred  also.)  The  steps  performed  by  the  user  are  as  follows:  First 
an  envelop  instance,  s,  is  created.  In  doing  so  the  user  specifies  the  destination  sta¬ 
tion  s2.  Next  the  user  puts  x  into  e  and  e  into  o2,  the  out-tray  of  station  si  (see  Figure 
5).  The  remaining  steps  of  the  transfer  are  performed  by  the  OIS.  This  example  illus¬ 
trates  how  closely  operations  within  the  model  resemble  those  that  would  be  per¬ 
formed  in  a  paper-based  system. 

In  order  for  a  user  to  'maiir  an  object  to  a  second  user  it  is  only  necessary  to 
know  the  object  to  be  transferred  and  the  destination  station.  These  two  objects  may 
be  selected  by  the  general  method  of  specif5dng  selection  conditions  on  property 
values  (it  would  be  useful  for  station  to  have  a  property  identifying  the  users  of  partic¬ 
ular  station  instances). 


3,3  Templates 

Until  now  we  have  been  considering  objects  and  their  properties  at  the  data¬ 
logical  level.  The  transformation  to  the  infological  level  involves  the  use  of  templates 
[23].  A  template  is  a  structure  which  provides  an  interpretation  for  an  object.  Sup¬ 
pose  X  is  an  object  and  T  a  template.  We  may  symbolically  represent  the  interpretation 
of  X  under  T  as  T(2).  The  purpose  of  the  transformation  T(z)  is  to  present  x  to  the  user 
in  an  understandable  form.  For  example,  instead  of  presenting  the  user  with  a  simple 
list  of  z's  property  values,  we  may  wish  to  provide  additional  information  such  as  how 
the  property  values  are  related  and  what  operations  may  be  performed  on  these 
values.  Of  course  it  cannot  be  guaranteed  that  T(z)  is  any  more  understandable  than  x 
itself.  The  success  with  which  T  provides  adequate  interpretations  depends  upon  the 
definition  of  T. 

A  template  type  is  an  aggregate  object  type  used  for  representing  templates. 
Let  T  be  a  template  type  and  x  an  arbitrary  object.  If  x  part-of  t  holds  with  t  an 
Instance  of  T,  then  we  say  T(x)  =  t.  So,  in  order  to  interpret  an  object  it  is  only  neces¬ 
sary  to  include  the  object  as  a  constituent  of  some  template  instance.  As  with  other 
aggregate  objects,  constraints  may  be  specified  which  determine  the  allowable  consti¬ 
tuents.  Viewing  T  as  a  function,  these  constraints  determine  the  domain  of  T. 

With  the  exception  of  audio  properties,  property  values  are  visually  presented 
to  the  user.  A  template  describes  the  positioning  of.  property  values  in  the  visual  field. 
In  general,  a  template  definition  consists  of: 

1)  boxes  (which  may  be  nested), 

2)  positioning  constraints  between  boxes, 

3)  a  mapping  mechanism  which  associates  property  values  with  boxes, 

4)  a  specification  of  the  allowed  operations  upon  the  contents  of  the  various 

boxes, 

5)  'background'  information. 
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6)  an  icon. 

Specific  regions  within  the  visual  field  are  referred  to  as  boxes  [7].  A  template  is  a 
hierarchy  of  boxes.  Property  vadues  are  displayed  within  boxes,  as  is  any  'background' 
information  (this  is  information  which  is  the  same  for  all  instances  of  a  template  type). 
Each  box  has  a  number  of  operations  which  can  be  used  to  manipulate  the  contents  of 
the  box.  A  graphic  symbol  known  as  an  icon  can  be  used  to  identify  the  template  type. 

Examples  of  templates  are  shown  in  Figures  6  to  9.  The  order-template  (Fig¬ 
ure  6)  would  be  used  to  display  instances  of  type  ordei^orm.  The  positions  for  the 
date,  total  and  item  property  vedues  are  indicated  on  the  template.  Similarly  Figures  7 
and  8  show  templates  to  be  used  with  letter  and  employee^record  objects.  The  opera¬ 
tions  associated  with  the  various  boxes  of  these  templates  are  not  shown,  but  would 
include  such  things  as  editing  and  scrolling.  When  the  user  selects  a  particular  box,  a 
menu  of  the  available  operations  can  be  displayed.  In  some  cases  it  is  useful  to  expli¬ 
citly  show  the  operations  associated  with  the  box.  Consider  the  template  in  Figure  9, 
which  is  intended  to  be  used  with  the  following  object  type; 

voice-message  — >  to  from  date 
subject  message  reply 
to  — >  character 
from  — >  character 
date  ->  character 
subject  — >  character 
message  — >  audio 
reply  —  >  audio 

Because  of  the  ephemeral  nature  of  sound,  it  is  difficult  to  know  just  when  to  output 
audio  property  values.  Here  the  decision  is  left  to  the  user.  In  voice^nessage-tempd.ate 
the  boxes  which  correspond  to  the  audio  valued  properties  message  and  reply  are 
menus.  By  selecting  a  menu  command  the  user  can  control  the  output  of  the  associ¬ 
ated  audio  property  value. 

Often  it  is  useful  to  identify  the  type  of  an  object  without  displaying  the  object 
in  all  its  detedl.  For  this  purpose  each  template  has  a  graphic  valued  property  used  for 
an  icon.  For  example,  the  icon  for  envelope  could  be  a  small  rectangle  with  a  'stamp' 
on  it.  the  icon  for  a  file  aggregate  type  could  resemble  a  file  cabinet.  The  operations 
which  can  be  performed  upon  an  icon  are  those  eissociated  with  the  outermost  box  of 
the  template.  These  operations  include  such  things  as  displaying  the  complete  tem¬ 
plate  and  moving  the  object  contained  within  the  template.  This  use  of  icons  is  similar 
to  that  found  in  the  Star  system  [18]  and  spatial  data  management  systems  [9]. 

Templates  have  many  uses  in  addition  to  formatting  and  the  support  of  user 
interaction.  One  such  additional  function  is  to  provide  different  views  of  an  object, 
given  templates  T1  and  T2  and  an  object  x.  then  Tl(x)  may  be  quite  different  from 
T2(x),  For  example,  two  separate  departments  may  use  different  templates  for  the 
same  form  t3^e.  Templates  have  also  been  used  in  the  specification  of  queries  [24,  25] 
and  office  procedures  [24], 


4.  Conclusion 

This  paper  has  outlined  an  office  information  model  and  discussed  the 
representation  of  objects  and  operations  within  the  model.  The  model  has  a  number  of 
interesting  features:  First,  data  values  are  not  restricted  to  the  standard  boolean. 
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character  and  numeric  types;  the  model  allows  audio  and  graphic  data.  Secondly,  a 
number  of  concepts  from  the  field  of  semantic  data  modelling  have  been  used.  These 
include  the  abstraction  mechanisms  of  aggregation  and  generalization  and  the  use  of 
objects.  A  similar  approach  has  previously  been  suggested  for  the  design  of  end  user 
facilities  [ll].  The  approach  used  in  this  paper  is  to  represent  objects  such  as  stations 
and  files  within  the  model  itself.  Finally,  the  model  contains  interpretative  constructs, 
known  as  templates,  which  aid  the  user  in  understanding  the  system. 

The  objects  and  operations  described  in  this  paper  are  relatively  simple. 
More  complex  aspects  of  the  office,  such  as  office  procedures,  involve  operations  and 
constraints  which  are  difficult  to  formulate  within  the  model  as  they  make  use  of  appli¬ 
cation  specific  knowledge.  One  possible  direction  for  future  work  is  to  try  end  include 
such  knowledge  in  the  model. 
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ABSTRACT 

Message  management  systems  are  an  integration  of 
computer-based  message  systems  and  database  mauiagement  sys¬ 
tems.  Structure  is  imposed  on  the  messages  so  that  the  system 
cein  manage  the  messeiges  for  the  users.  Message  management  sys¬ 
tems  have  some  unique  characteristics.  A  model  for  these  systems 
is  proposed.  A  representation  of  the  model  using  Smalltalk-like 
objects  is  also  discussed. 


1.  Introduction 

Communication  is  one  of  the  fundamental  human  activities  and  is  vital  to 
the  running  of  a  business  or  organization.  Computer-Based  Message  Systems 
(CBMS’s)  are  a  relatively  new  communication  medium  where  computers  and  net¬ 
works  are  combined  into  a  powerful  facility.  They  have  the  potential  to 
significantly  reduce  the  amount  of  resources  now  used  by  organizations  for  com¬ 
munication  and  to  revolutionize  our  methods  of  interpersonal  communication 
[BAl]. 

As  the  use  of  CBMS’s  increases,  more  facilities  will  be  demanded  of  them 
besides  just  the  delivery  of  messages.  Currently  there  are  "value-added"  sys¬ 
tems  that  provide  such  services  as  text-editing  for  message  creation  and  filing 
systems  for  message  filing  [KIR].  In  addition.  CBMS's  must  be  able  to  do  some- 
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thing  "intelligent"  with  the  messages,  like  sorting  messages  according  to  tj^je, 
contents  or  originator,  filing  incoming  messages,  preparing  stock  answers,  main¬ 
taining  a  calender,  or  systematically  querying  the  messages  on  file  [ELL].  That 
is,  CBMS's  must  be  able  to  Tnanage  messages  as  well  as  deliver  them. 

Conceptually,  a  message  management  system  is  the  combination  of  a  CBMS 
and  a  DBMS  (Data  Base  Management  System).  By  integrating  the  two  system 
capabilities  we  obtain  a  system  which  partially  interprets  the  messages  it  han¬ 
dles.  Users  can  query  the  message  system  to  find  messages,  can  accumulate 
data  that  the  messages  contain  and  can  specify  automatic  processing  and  rout¬ 
ing  procedures  for  the  messages.  Several  existing  systems  have  incorporated 
some  of  the  features  of  a  message  management  system  [VlT,BRO,TSIb]  but  they 
are  very  different  in  their  functioning  and  their  characteristics. 

An  important  prerequisite  to  the  study  and  eventual  design  and  implemen¬ 
tation  of  message  management  systems  is  a  model  that  accurately  captures  the 
main  properties  of  the  systems.  Traditional  Data  Base  models  are  insufficient  for 
several  reasons; 

[1]  Data  Base  models  are  intended  to  represent  a  very  structured  and  con¬ 
trolled  world  as  in  data  processing.  The  communication  in  an  office  is  not 
rigidly  structured,  but  flexible,  and  in  some  aspects  informal  [HIL.UHLj. 

[3]  Message  management  systems  emphasize  local  data  and  the  concept  of 
ownership  of  data.  Data  Base  models  only  supply  a  global  notion  of  data. 
Also,  they  are  not  able  to  represent  the  target  of  a  message,  except 
perhaps  as  an  attribute, 

[3]  Side  effects  are  present,  and  even  desirable,  in  message  management  sys¬ 
tems  (e.g.  the  generation  of  an  alert  when  a  message  is  created  or 
updated).  Traditional  Data  Base  models  attempt  to  eliminate  side  effects. 
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[4]  Message  management  systems  contain  a  lot  of  activity  (e,g.  sending,  receiv¬ 
ing  and  responding  to  messages);  what  the  actions  are  and  where  they  talce 
place  are  very  important  properties  that  must  be  modelled.  Data  Base 
models  do  not  emphasize  the  actions  taken  by  the  system,  only  the  entities 
and  their  relationships.  In  addition.  Data  Base  models  do  not  represent  the 
fact  that  some  of  the  relationships  in  a  message  maneigement  system  are 
dependant  upon  the  physical  location  of  the  data. 

Current  message  system  models  do  not  represent  many  message  types  and 
limit  message  structure  to  a  uniform  set  of  fields  [BOCH.KIR.DEU].  In  addition,  a 
problem  solving  plays  a  major  role  in  office  activity.  Some  kind  of  procedural 
model  is  required  to  represent  this  characteristic. 

2,  The  Model 

We  view  message  management  systems  according  to  the  framework 
presented  in  [TSIa].  We  develop  the  model  and  discuss  how  it  is  applied  to  this 
framework.  We  first  give  a  brief  description  of  the  objects  used  to  represent  the 
model. 

The  objects  in  our  model  are  derived  from  Smalltalk  [BYTE]  which  is  an 
object-oriented  language,  i.e.  meaning  remains  with  the  objects  of  the  system 
and  code  remains  an  abstract  form  directing  the  fiow  of  communication.  An 
object  has  an  internal  state  and  responds  to  a  set  of  requests  or  signals.  A 
response  to  a  request  is  implemented  by  a  method  which  does  one  of  the  follow¬ 
ing:  changes  the  internal  state  of  the  object:  transmits  signals  to  other  objects, 
or  accepts  or  provides  information.  The  internal  structure  of  a  receiving  object 
is  hidden  from  the  originator  of  a  request;  the  originator  only  needs  to  know  the 
receiver’s  signal  set. 

Every  object  is  created  as  an  instance  of  a  class.  The  class  holds  the 
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detailed  representation  of  its  instances,  the  signals  to  which  they  can  respond, 
and  the  methods  for  computing  the  appropriate  responses.  When  a  signal  is  sent 
to  an  instance,  that  instance  in  turn  requests  the  appropriate  method  from  its 
class.  The  method  returned  is  then  applied  to  the  arguments  of  the  signal, 

Classes  are  hierarchical,  A  supercleiss  is  used  to  describe  the  behaviour 
common  to  several  classes.  Subclasses  inherit  the  behaviour  of  the  superclass 
and  pass  it  on  to  their  instances. 

The  objects  of  our  model  also  have  capabilities  beyond  those  found  in 
Smalltalk,  The  objects  of  the  PIE  (Personal  Information  Environment) 
[GOLDa,b,c,d]  system  provide  us  with  three  capabilities  that  can  be  used  to 
model  important  properties  of  a  message  management  system.  These  are: 

[1]  MidtipLe  Perspectives:  the  assignment  of  more  than  one  point  of  view  that 
allows  inheritance  of  behaviour  from  independent  superclasses. 

[2]  Metadescription:  the  assignment  of  constraints  to  attributes  that  allows  the 
system  to  check  new  values  and  propogate  their  intended  effects.  A  more 
complete  facility,  called  contracts,  is  described  in  [GOLDb]. 

[3]  Identification:  the  assignment  of  a  system-wide  unique  identifier  that  allows 
multiple  users  to  manipulate  a  common  set  of  objects. 

Objects  were  chosen  as  a  basic  modelling  unit  because  of  their  applicability 
to  the  characteristics  of  message  management  systems.  Objects  capture  the 
active  and  procedural  aspects  of  message  management  systems.  Locality  and 
physical  independence  imply  a  need  for  a  modular  approach  and  for  auto¬ 
nomous  en'ities  which  naturally  leads  to  a  message  passing  model  based  on 
objects.  In  addition,  we  need  basic  categories  of  entities  which  implies  the  use  of 
classes. 


Objects  are  a  very  powerful  and  general  tool  for  representation  and  so  can 
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be  easily  adapted  to  our  more  specific  message  management  model.  We  also 
gain  the  advantage  that  an  implementation  of  objects  as  described  above  will 

s 

lead  to  an  implementation  of  a  message  management  system. 

We  use  four  basic  constructs  to  model  message  management  systems. 
These  are  messages,  message  sets,  message  types  and  eigents.  A  message  is  an 
instance  of  commumcation.  It  is  a  triple  <id,  scope,  body>,  where  "id  is  a  unique 
system-wide  identifier,  scope  is  the  set  of  addresses  affected  by  the  message,  i.e. 
the  potential  recipients  and  the  originator,  and  body  is  the  information  to  be 
communicated.  We  can  perform  the  following  operations  on  a  message;  create, 
delete,  update,  read,  send  and  copy. 

Each  message  is  represented  as  an  instance  of  a  class  of  objects,  MESSAGE, 
with  instance  variables  id,  scope  and  body  and  methods  to  perform  the  above 
operations. 

A  message  set  is  a  set  of  messages.  Membership  to  a  message  set  can  be 
defined  as  automatic  or  manual.  Automatic  membership  is  specified  by  a  predi¬ 
cate.  When  a  message  is  created  it  is  automatically  placed  in  a  message  set  if  it 
satisfies  the  associated  predicate.  Manual  membership  is  specified  by  enumera¬ 
tion.  i.e.  a  message  must  be  explicitly  placed  in  a  message  set.  A  message  set 
can  have  further  properties,  called  constraining  properties,  that  do  not  effect 
membership  but  cein  determine  subsets  within  a  message  set.  We  assume  every 
messcige  must  belong  to  at  least  one  message  set  while  it  exists. 

A  message  set  is  represented  by  an  object  class  MESSAGE  SET  which  is  a 
subclass  of  a  system  defined  class  SET  [BYTE].  Thus  it  will  inherit  methods  to 
perform  the  characterizing  operations  of  a  set  such  as  adding  and  deleting 
members  or  testing  for  membership. 

A  message  type  is  a  message  set  with  a  set  of  interpretation  Junctions.  The 
Interpretation  functions  impose  a  structure  on  the  body  of  all  messages  of  the 
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type.  Messages  can  be  created  as  a  member  of  a  type  or  made  one  at  a  later 
time,  i.e.  the  type  of  a  message  can  be  changed,  and  a  message  can  exist 
without  being  a  member  of  a  t3p>e. 

Message  tjpes  are  represented  as  object  classes.  They  eire  subclasses  of  the 
MESSAGE  SET  class,  and  so  inherit  the  methods  to  manipulate  messages.  The 
claisses  also  have  methods  to  implement  the  interpretation  functions  associated 
with  the  type. 

An  ag^Tit  is  an  entity  that  can  initiate  operations  on  messages  or  message 
sets,  such  as  creating  a  message  and  then  moving  it  from  one  message  set  to 
another. 

A  message  models  the  basic  unit  of  communication  in  a  mesage  manage¬ 
ment  system.  The  information  to  be  communicated  is  in  the  body  of  a  message 
and  the  identifier  is  used  to  keep  track  of  a  message  within  the  system.  A  mes¬ 
sage  can  be  associated  with  a  type  or  considered  "untyped". 

In  the  object-oriented  representation,  an  untyped  message  corresponds  to 
an  object  that  is  an  instance  of  class  MESSAGE  alone.  A  typed  message  also  has  a 
perspective  associated  with  the  class  representing  the  message  type. 

As  a  member  of  a  type  a  message’s  contents  are  given  meaning  according 
to  the  type's  interpretation  functions.  The  type  of  a  message  may  change  during 
its  lifetime.  Different  users  may  wish  to  view  the  contents  in  different  ways.  For 
example,  one  user  sends  a  message  consisting  of  a  table  of  numbers  to  two  other 
users.  One  recipient  may  wish  to  interpret  the  table  as  a  graph,  the  other  as  a 
histogram.  We  model  changing  the  interpretation  of  a  message  by  the  operations 
of  moving  or  cop3dng  the  message  from  one  type  to  another. 

A  message  can  be  declared  to  be  of  a  type  within  a  certain  category.  In  such 
a  case  the  message  will  inherit  the  behaviour  of  the  category.  For  example,  a 
message  can  be  created  as  type  BUSINESS  LETTER  within  a  category  LETTER.  It 
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will  inherit  all  the  characteristics  of  a  letter  plus  those  unique  to  business  letter. 

Categories  ere  captured  in  the  object  representation  by  the  inheritence  between 

* 

classes. 

The  scope  of  a  message  cem  affect  only  a  single  address  (private  informa¬ 
tion),  a  group  of  addresses  or  all  addresses  (public  information).  It  can  consist 
of  a  set  of  addresses  or  a  procedure  that  evaluates  to  a  set  of  addresses.  This 
procedure  can  be  evaluated  centrally  or  in  a  distributed  fashion. 

We  illustrate  our  model  with  an  example  from  a  stock  exchange.  In  a 
simplified  version,  there  eire  traders  on  the  floor  of  the  exchange  that  carry  out 
the  buying  and  selling  of  stocks  for  their  offices.  Tney  are  arranged  at  stations 
(perhaps  a  terminal  or  a  phone)  about  the  floor.  Each  station  must  be  con¬ 
stantly  manned  while  the  exchange  is  open,  so  more  than  one  trader  is  associ¬ 
ated  with  each  station. 

The  traders  are  directed  by  brokers  in  the  office  as  to  what  to  buy  and  sell. 
They  respond  by  carrying  out  the  transaction  (as  closely  as  possible)  and  then 
confirming,  with  the  broker,  what  was  done.  The  traders  may  also  pick  up  tips 
from  conversations  on  the  floor  which  they  relay  to  their  offices.  All  stock  infor¬ 
mation  (e.g.  quantity  available  and  price  of  each  stock)  is  maintained  in  a  data¬ 
base  and  must  be  updated  whenever  transactions  are  carried  out  or  changes 
occur.  All  brokers  and  traders  are  notified  of  any  changes  to  the  database. 

The  communication  in  this  example  is  modelled  by  messages  of  four  types  - 
DIRECTIVE,  REPLY.  MEMO  and  STOCK  UPDATE.  Messages  of  type  DIRECTIVE 
represent  the  communication  from  the  brokers  to  the  traders  informing  them  of 
the  transactions  to  be  carried  out.  Each  message  corresponds  to  one  transac¬ 
tion  and  contains  the  attributes  type  (buy  or  sell),  stock,  amount  and  special 
instructions.  Messages  of  type  REPLY  represent  the  traders  confirmation  of 
transactions  and  contain  attributes  which  relate  to  the  original  directive  and 
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describe  what  was  done.  Messages  of  type  MEMO  capture  communication  carried 
out  at  a  more  personal  level.  They  are  used  when  traders  msh  to  pass  on  infor¬ 
mation  to  the  brokers  or  when  information  is  to  be  shared  among  several  peo¬ 
ple.  Each  message  contains  attributes  that  correspond  to  fields  on  a  paper 
memo  -  to,  from,  subject  and  some  text.  Messages  of  type  STOCK  UPDATE 
represent  communication  with  the  stock  database.  They  are  created  when 
changes  to  information  in  the  database  are  required  and  contain  the  attributes 
type  (update,  Insert,  delete),  stock  and  amount. 

The  carnTnunication  base  is  the  medium  where  users  can  add  and  obtain 
information  in  a  message  management  system.  It  is  the  repository  for  all  mes¬ 
sages  and  the  medium  of  communication  in  the  system.  ’'Ne  model  the  communi¬ 
cation  base  as  a  message  set  that  contains  all  messages.  Each  message  is 
automatically  made  a  member  of  the  message  set  when  created  and  must 
remain  a  member  until  it  is  deleted.  We  represent  a  communication  base  as  an 
object  of  class  COMMUNICATION  BASE  which  is  a  subclass  of  MESSAGE  SET  and 
provides  a  perspective  on  all  messages. 

Every  operation  on  the  communication  base  is  issued  by  an  agent.  We  use 
an  agent  to  model  a  user  or  process,  such  as  an  automatic  procedure  [HOGG], 
acting  on  a  user’s  behalf.  An  agent  accesses  the  communication  base  through  an 
address. 

If  we  consider  an  office,  correspondence  is  not  necessarily  addressed  to,  or 
handled  by,  a  single  person;  for  example,  messages  may  be  addressed  to  the 
manager  of  the  Systems  Department  but  both  the  manager  and  his  secretary 
may  handle  them.  Similarly,  one  person  may  handle  correspondence  for 
different  addresses;  for  example,  a  person  may  be  in  charge  of  two  projects  and 
receive  mail  addressed  to  the  project  leaders.  Thus,  we  can  consider  mail  to  be 
sent  to  a  role  in  the  office. 
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We  define  an  office  role  to  be  the  set  of  actions  and  reponsibilities  associ¬ 
ated  with  a  particular  office  function.  "Employee",  "Manager".  "Chief  Program- 
mer"  are  all  examples  of  office  roles.  Within  this  definition  we  must  allow  a  per¬ 
son  to  play  the  role  of  himself,  to  accomodate  personal  mail. 

An  address  in  a  message  management  system  can  correspond  to  am  office 
role.  A  user  is  then  explicitly  linked  to  a  role  in  order  to  have  access  to  its  mes¬ 
sages.  The  sending  of  messages  to  roles  provides  the  flexibility  for  users  to  share 
a  role  and  to  participate  in  several  roles. 

Besides  an  office  role,  an  address  in  a  message  management  system  can 
correspond  to  two  other  types  of  entities.  An  address  can  be  an  interface 
between  the  message  management  system  and  another  computer  system.  A 
user  can  direct  a  request  for  information,  via  a  message,  to  an  address  associ¬ 
ated  with  another  computer  system  which  can  process  the  request  and  send  a 
response  back  to  the  user. 

An  address  in  the  system  can  also  relate  to  a  system-dependant  entity  that 
reflects  the  structure  of  the  system  or  office.  It  exists  to  supply  direction  to 
messages  travelling  about  the  system.  For  instance,  we  may  wish  to  convey 
information  to  a  certain  department,  but  are  not  sure  who  is  the  appropriate 
person  within  the  department  to  deal  with  the  information.  We  require  an 
address  that,  upon  receipt  of  a  message,  can  choose  the  correct  role  to  handle 
the  message,  depending  upon  the  t3p>e  or  contents  of  the  message  or  the  state 
of  the  system,  and  then  send  the  message  on  to  this  address. 

An  address  has  two  components:  a  logical  and  a  physical.  The  Logical 
address  is  the  identifier,  or  set  of  identifiers,  employed  by  users  to  access  the 
address.  A  possible  representation  is  a  set  of  <attribute, value >  pairs  [KER].  The 
physical  address  is  the  actual  physical  location  where  the  local  datagrams  are 
stored.  We  can  consider  this  as  being  a  mailbox(es)  on  a  station(s)  [TSIb].  Users 
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■will  generally  employ  the  logical  address  and  the  system  "will  have  to  map  it  to 
the  physical  address(es).  in  either  a  central  or  distributed  fashion.  The  binding 
between  the  logical  and  physical  addresses  should  be  sufliciently  flexible  so  that 
(l)  users  can  sign  on  at  any  physical  location  and  still  be  able  to  access  their 
portion  of  the  communication  base;  (2)  more  than  one  logical  address  can  be 
associated  with  more  than  one  physical;  (3)  the  binding  is  easily  modified,  and 
(4)  the  binding  can  be  temporary  and  active,  e.g,  the  physical  address  can  be 
temporarily  changed  and  messages  arriving  at  the  old  address  automatically 
forwarded  to  the  temporary  one. 

In  our  message  management  model,  an  address  is  represented  as  a  mes¬ 
sage  set.  The  messages  local  to  an  address  at  any  particular  time  are  members 
of  the  message  set.  A  message  automatically  becomes  local  to  the  address 
where  it  is  created.  Any  messages  received  from  other  addresses  (moved  or 
copied  to  the  address'  message  set)  are  considered  local  until  they  are  sent  or 
deleted.  The  set  of  address  message  sets  forms  a  partition  of  the  messages  in 
the  communication  base.  A  messaige  must  be  local  to  some  address  to  exist.  We 
represent  an  address  as  an  object  of  the  class  ADDRESSES,  a  subclass  of  MES¬ 
SAGE  SET. 

The  addresses  in  the  stock  exchange  are  each  trader  station,  each  indivi¬ 
dual  trader,  each  broker  and  the  stock  database.  At  any  one  time  a  trader  has 
access  to  his  personal  address  and  to  a  station  address.  The  stock  database 
address  can  correspond  to  an  external  (to  the  message  management  system) 
database  system.  Messages  received  at  that  address  are  translated  into  data¬ 
base  operations  and  presented  to  the  system  to  be  performed.  When  the  address 
Is  notified  that  the  operations  are  completed  it  generates  messages  of  type 
MEMO  to  notify  the  traders  and  brokers  of  the  changes. 

Associated  with  the  scope  is  the  optional  requirement  of  ein  alert.  The 
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agents  operating  through  an  address  contained  In  the  scope  of  a  message  are 
guaranteed  to  receive  the  alert.  They  may  or  may  not  receive  the  message  as 
they  choose.  We  represent  an  alert  as  a  method  associated  with  the  class  MES¬ 
SAGE  that  generates  a  signal  containing  the  message  object’s  id  instance  vari¬ 
able. 

A  view  is  the  dual  concept  of  a  scope.  It  serves  as  a  filter  for  the  messages 
affecting  an  address  or  a  particular  agent,  A  view  may  select  messages  accord¬ 
ing  to  originating  address,  type  or  even  contents.  A  view  can  be  imposed 
between  an  address  and  the  rest  of  the  system  or  between  an  address  and  a  sub¬ 
set  of  its  associated  agents.  Thus  all  users  playing  a  role  do  not  have  to  be 
treated  equally.  An  important  case  of  a  view  is  an  active  view.  They  act  as 
automatic  procedures  and  produce  side  effects  such  as  collecting  messages,  for¬ 
warding  messages,  modifying  messages  or  generating  new  messages. 

A  view  is  again  modelled  as  a  message  set.  Its  members  are  those  messages 
that  satisfy  the  view’s  requirements.  These  requirements  are  expressed  as 
defining  predicates  and  membership  to  the  message  set  will  be  automatic.  The 
active  portion  of  a  view  is  modelled  by  an  agent  associated  with  the  message  set 
that  operates  on  its  members.  We  represent  a  view  as  a  subclass  of  MESSAGE 
SET. 

The  stock  exchange  example  can  provide  several  instances  of  a  view.  A  view 
could  be  set  up  at  a  trader  station  that  looks  for  a  particular  type  of  stock  in  the 
incoming  directives  and  gives  them  the  highest  priority.  A  view  could  also  be  set 
up  that  collects  all  transactions  for  particular  stocks  and  only  notifies  the 
traders  at  the  station  when  a  certain  dollar  amount  is  reached.  Views  could  also 
be  imposed  between  certain  traders  and  their  station  to  allow  them  to  specialize 
and  only  handle  transactions  for  certain  types  of  stocks. 

We  can  "send”  a  message  in  one  of  three  different  ways.  In  the  first  case,  a 
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copy  of  the  messaige  is  created  for  each  recipient  and  placed  in  their  address. 
From  then  on  each  copy  is  independent  and  treated  as  a  separate  message.  In 
the  second  case,  there  is  one  logically  common  recording  of  the  message  for 
everyone  in  the  scope.  Information  is  communicated  by  modifying  the  common 
recording.  Everyone  in  the  scope  is  sent  an  alert  when  the  state  of  the  message 
is  changed  and  they  must  then  read  the  message.  Finally,  in  the  third  case,  a 
message  is  recorded  only  until  it  is  received  by  the  recipients  and  then  it  disap¬ 
pears,  If  the  recipients  want  to  retain  the  information  they  must  explicitly  make 
a  copy  after  they  receive  the  message. 

We  can  represent  sending  a  message  by  the  exchange  of  signals  between 
address  objects.  In  one  case,  a  copy  of  the  message  is  an  argument  of  the  signal 
from  the  originating  address  and  placed  directly  in  a  receiving  address’  local 
message  set.  In  another  case,  only  the  message’s  id  is  passed  as  an  argument 
and  the  recipient  must  then  pass  a  signal  back  requesting  a  copy  of  the  mes¬ 
sage. 

3.  Conclusions 

Our  model  for  a  message  management  system  has  several  advantages  over 
current  models  for  CBMS's; 

[1]  Our  model  is  concerned  with  more  than  just  the  delivery  of  messages.  It 
Includes  facilities  to  actually  manage  the  messages  and  tailor  the  system  to 
meet  a  specific  user's  needs. 

[2]  Our  model  can  represent  a  greater  variety  of  entities  represented,  such  as 
scopes  and  views,  and  allow  for  the  existence  of  automatic  procedures. 

[3]  Our  model  allows  for  types  of  messages  rather  than  one  universal  structure. 

[4]  Our  model  represents  both  present  and  future  communication  (datagrams 
stored  for  later  delivery). 
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[5]  Our  model  captures  the  concepts  of  both  local  and  global  data. 

We  also  present  a  very  flexible  definition  of  address.  Users  can  be  easily 
linked  to,  and  removed  from,  roles.  They  can  play  more  than  one  role  and 
several  users  can  play  a  single  role,  A  role  can  be  associated  with  one  or  more 
logical  addresses  and  one  or  more  physiced  addresses. 

This  defimtion  of  address  accomodates  temporary  changes  to  the  physical 
addresses  associated  with  a  role  so  that  a  user  can  always  receive  his  mail  even 
if  he  changes  location.  It  is  an  implementation  question  whether  this  facility 
should  actually  be  part  of  the  address  or  be  provided  by  imposing  a  view. 

Users  are  also  provided  with  the  option  of  establishing  a  priority  of  the 
users  playing  a  role.  One  user  may  be  designated  to  prescreen  all  messages  and 
then  forward  them  to  other  users.  Users  may  be  allowed  to  receive  certain  types 
of  messages  and  restricted  from  receiving  others. 

One  problem  with  this  definition  of  address  is  that  with  users  sharing  a  role 
the  possibility  arises  that  a  user  may  be  accessing  an  address  that  is  not  on  his 
station.  This  means  higher  communication  costs.  We  can  reduce  the  communica¬ 
tion  costs  by  keeping  duplicate  data  at  several  stations  but  we  incur  higher 
storage  costs  and  require  the  system  to  maintain  consistency  among  the  copies. 
These  performance  questions  are  the  same  as  those  found  with  distributed  data¬ 
bases  and  techniques  used  there  are  applicable.  It  is  not  clear  how  frequently 
instances  of  users  sharing  a  role  across  physical  station  boundaries  would  occur 
and  if  it  poses  a  significant  performance  problem. 

The  object  representation  captures  the  important  properties  of  locality  and 
physical  independence  because  of  the  modularity  of  objects  and  the  message¬ 
passing  between  them.  The  basic  categories  of  objects  were  included  to  make 
the  model  more  meaningful  and  help  our  understanding  of  the  systems.  But  in 
doing  so  we  restrict  the  model’s  applicability  to  only  message  management 
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systems.  The  object  representation  can  not  capture  the  notion  of  an  automatic 
procedure  and  hence  active  scopes  and  views. 

Our  message  management  model  provides  a  rich  and  flexible  communica¬ 
tion  environment.  Its  categories  of  objects  can  adequately  represent  a  fairly 
complex  situation,  as  can  be  seen  from  the  example.  The  model  will  also  serve 
as  a  starting  point  for  more  detailed  research  on  the  concepts  of  scope  aind  con¬ 
sistency  of  messages. 
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ABSTRACT 

A  message  management  system  provides  users  with  a  facility 
for  automatically  handling  messages.  This  paper  describes  a  tech¬ 
nique  for  characterizing  the  behaviour  of  such  a  system  in  terms  of 
message  flow.  Messages  may  be  conveniently  classed  according  to 
what  path  or  sequences  of  stations  they  visit.  Complicated  or 
unpredictable  behaviour  may  be  modeled  non-deterministicaily, 
and  the  resulting  message  paths  are  shown  to  be  regular  expres¬ 
sions. 


1.  Introduction 

Until  recently  messages  have  been  treated  as  a  passive  means  of  communication. 
Users  or  programs  could  exchange  information  by  passing  a  message  to  an  electronic 
mall  system,  which  would  guarantee  delivery  of  the  message  to  the  intended  recipient. 
Such  messages  typically  consist  of  a  header  and  a  body.  The  message  system  would 
examine  the  header  to  determine  the  target  and  deliver  the  body  of  the  message  intact 
without  interpretion.  The  comtents  of  a  message  would  be  of  no  concern  to  the  mes¬ 
sage  system.  In  addition,  a  message  system  provides  users  with  some  editing  capabil¬ 
ity  and  some  way  to  file  arriving  messages.  The  requirements  of  an  Office  Information 
System  [ElNu79],  however,  imply  that  a  much  more  powerful  message  handling  facility 
is  needed  in  an  office  environment.  Office  behaviour  tends  to  be  event-driuen  [AtBD79, 
FiHeBO,  MorgSO].  The  cu-rival  of  a  message  may,  for  example,  cause  one  or  more  pro¬ 
cedures  to  be  invoked.  Workstations  should  be  able  to  perform  some  of  these  pro¬ 
cedures. 
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A  message  manageTnent  system  provides  users  with  a  means  for  automatically 
processing  messages.  Most  office  procedures  are  semi-structured  [HaSiBO],  so  such  a 
system  would  enable  users  to  describe  tasks  which  follow  some  pattern,  but  may  turn 
to  the  user  at  critical  points  for  help.  Tasks  would,  for  example,  coordinate  messages, 
perform  routine  transformations  (evaluate  calculations  or  database  queries),  send 
reminders,  answer  queries  about  the  messages  it  has  seen,  and  draw  the  user’s  atten¬ 
tion  when  unusual  situations  arise.  Tasks  may  be  expressed  in  terms  of  SBA  boxes 
[deJoBO],  actors  [Hewi77],  data  frames  [EmblBO],  automatic  procedures  [TRGHBl]  or 
Smalltalk  objects  [BYTEBl]. 

One  way  of  incorporating  these  features  is  to  assume  that  messages  have  a  deep 
structure,  not  necesssirily  fixed,  but  certainly  something  richer  than  the  usual  amor¬ 
phous  body  of  text.  Tasks  may  interpret  messages  by  identifying  significant  ranges  of 
numeric  fields  or  finding  patterns  in  text  fields.  The  interpretation  may  result  in  a 
transformation  of  the  message,  routing  of  the  message  to  another  station,  or  some 
local  operation 

Since  tasks  may  be  triggered  without  user  intervention,  it  is  important  to  be  sure 
that  they  are  doing  what  is  expected  of  them.  An  unfortunate  configuration  of  indepen¬ 
dent  tasks  may  result  in  anomalous  behaviour  or  poor  performance.  Consider  the  case 
of  a  task  that  simply  forwards  mail  for  a  worker  on  holidays.  Two  such  tasks  may 
exchange  messages  all  day  without  anyone  noticing.  We  may  expect  a  sophisticated 
message  management  system  to  perform  message  flow  analysis,  task  analysis  and  sys¬ 
tem  instrumentation  [Ruli79].  A  model  which  supports  such  analysis  must  be  capable 
of  representing  task  inputs  and  outputs,  sources  and  destinations,  timing  constraints, 
the  amount  of  effort  spent  during  each  stage,  etc.  [SSKHBl]. 

In  this  paper  we  will  consider  the  simple  case  of  a  network  of  stations  that  com¬ 
municate  by  sending  messages.  Each  station  may  scan  an  incoming  message,  interpret 
it  by  initiating  any  number  of  actions,  possibly  modifying  its  contents,  and  then  either 
discard  it  or  forward  it  to  some  other  station  It  is,  of  course,  possible  to  model  more 
complicated  tasks  by  defining  substations  within  a  station. 

In  order  to  relieve  ourselves  of  as  much  detail  as  possible,  we  relate  correct 
behaviour  to  flow.  Messages  can  be  separated  into  cleisses  according  to  what  sequences 
of  stations  they  may  visit.  These  sequences  tell  us  not  only  what  routing  takes  place 
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but  also  the  tranformations  that  are  performed  iand  the  order  of  the  actions  that  take 
place.  The  model  presented  provides  a  tedhnique  for  determining  what  these 
sequences  are,  deciding  whether  or  not  they  may  terminate,  and  establishing  what 
end-conditions  hold  if  and  when  they  do  .terminate. 

8L  Tbe  message  flow  model 

A  message  flow  model  F=<S ,X,A,P>  consists  of  a  set  5  =  |si,S2 of  stations, 

a  set  X  of  messages,  and  relations  A:S»<X-^X  and  P:SxX-*S .  A  and  P  are  the  action 
and  roxiUng  relations.  They  need  neither  Ibe  well-  nor  totally-defined,  but  they  are 
often  assumed  to  be  functions. 

A  message  x^X  starting  at  some  station ‘se5  will  trace  a  path  through  S  by 
repeated  applications  of  A  and  P.  'If  we  define  ^(s,x)=P(s,A(s.x))  and 
//{s,ir)=(^(s,a:),/4(s,a;)),  then  we  follow  the  path 

s,  ^(s.x),  Q{N{s ,x)),  Q{N^{s ,x)),  •  *  •  .  We  can  recursively  define  the  path  of  a  mes¬ 
sage  x^X  starting  in  station  sg5  to  be  the  string  in  S*  obtained  by 
(p{s  ,x)=s <p{N {s  ,x)) .  If  A  and  P  are  functions,  or  at  least  totally  defined,  then  this  path 
will  clearly  never  terminate.  This  is  also  true  of  a  message’s  history, 
^{s ,x)={s ,x)^{N {s ,x))  in  {S>^X)*.  A  history  records  not  only  the  sequence  of  stations 
that  a  message  visits,  but  also  the  contents  that  it  had  at  the  time. 

For  a  given  message  we  must  designate  certain  stations  as  initial  or  final.  To  this 
end  we  augment  our  set  S  with  the  special  stations  a  and  cj.  A  station  s  is  an  initial 
station  for  x  if  P{a,x)=s,  and  s  is  a  final  station  for  x  if  P{s,x)=cj.  x  is  created  by  a 
user  or  a  procedure  at  s  if  s  is  initial,  and  is  discarded  or  archived  if  s  is  final.  Now  A 
and  P  may  be  totally  defined  and  our  message  ipattis  may  terminate  when  a  message 
reaches  its  corresponding  final  station,  by  letting  93((i;,x)=$(ii;,x)=e,  the  empty  string. 
Certain  obvious  conditions  must  hold:  A(a,x)=r,  A(o,x)=x.  P{cj.x)=cj  and  V 
s^S  P{s,x)i>^a. 

Determining  which  paths  a  message  may  take  is  complicated  by  the  changes  of  its 
contents.  A  message  reaching  some  final  station  may  contain  contents  different  from 
the  original  contents.  Nevertheless  we  attempt  to  approach  message  flow  by  consider¬ 
ing  classes  of  messages  which  may  follow  the  same  paths,  rather  than  by  solving  these 
paths  individually. 
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At  this  point,  we  will  make  the  assumption  that  messeiges  are  of  the  form,  at  least 
for  routing  purposes,  of  a  cross  product  where  each  is  an  attribute.  Mes- 

’  i  =  l 

sages  whose  body  have  no  structure  would  have  a  header  {Xi.Xz . J^-i)  and  body 

Xj^.  We  assume  for  the  moment  that  messages  are  of  fixed  type  with  no  repeating 
fields.  Attributes  which  directly  and  indlractly  aflect  routing  are  called  raiding  and 
coTiirof  attributes  respectively.  If  the  routing  relation  can  be  expressed  in  terms  of 
only  simple  conditions  Involving  the  routing  attributes,  and  If  actions  set  control  attri¬ 
butes  to  constants  after  checking  only  simple  conditions,  then  the  collection  of  simple 
conditions  provides  a  partition  of  each  control  attribute  in  a  natural  way  [TsicBl], 

If  x  =  {x^,X2 . r*. . x'={x^.x2 . x\ . x„).  and 

P{s  ,x)i^P{s  ,x'),  then  Xk  is  a  roubing  attribute.  X^  may  also  affect  routing  only  after 
one  or  more  applications  of  A  and  P.  This  is  possible  if  A*  is  used  to  set  the  value  of 
some  routing  attribute.  We  call  X^  a  control  attribute  if  3seS,i€N-9- 
.Q{N*{s.x))^Q{/Pis,x^). 

We  let  Pyj  be  the  predicate  P(s^,x)=Sj-.  Consider  such  predicates  of  the  form 
■  Cn\  where  each  C*  is  a  conjunction  of  simple  conditions  Xk<op>u. 
<op  >^\  =  ,9^  ,<,^,>,^1  andueAf*.  Consider  also  the  action  A*  at  stations  which  sets  x  to 
A(s,x).  If  the  component  of  which  modifies  a  control  attribute  X^  can  be  encoded  as 
if  vICjA-  •  ■  Cnl  then  set  Xk=constant  where  the  Cjt's  are  again  simple  conditions, 
then  the  u's  of  those  simple  conditions  induce  a  natural  partition  Hjt  of  any  control 
attribute  X^  into  ranges.  If  Xi  is  not  a  control  attribute,  we  simply  let  ni=|A!ij.  Note 
that  we  may  choose  not  to  model  such  A^’s,  since  they  contribute  no  information  about 

message  flow.  We  thus  obtain  a  partition  11=]^ Hi  of  X.  If  there  are  distinct  sim- 

<=i 

pie  conditions  on  Xi  to  be  found  in  all  the  A^,'s  and  Py 's.  then  Hi  partitions  Xi  into  Mi 
ranges.  So,  jilt  ]  =il/i  auid  | HI 

1=1 

We  denote  the  block  ni  of  n=^rri . containing  a  message  xeX  by  the 

equivalence  class  x  or  [x].  If  A  and  P  are  functions  as  described  with  the  above  res¬ 
trictions,  then  the  extensions  A(s,x)=[A(s,x)]  and  P{s .T)=P{s  .x)  are  weU-defined 
since  all  messages  In  the  class  S  satisfy  the  same  simple  conditions  on  routing  or  con¬ 


trol  attributes. 
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Let  5xn  be  the  set  of  states  of  F  with  respect  to  the  partition  IT.  We  then  obtain 
/i/':5'xn->5xn.  This  means  that  all  messages  In  a  particular  class  x  visit  precisely  the 
same  stations,  and  furthermore  are  all  mapped  to  the  same  new  class  by  any  transfor¬ 
mation  they  undergo. 

This  is  still  true  if  the  Q’s  are  more  general,  {eg.  search  for  a  pattern  in  text),  but 
now  the  partitions  may  be  as  large  as  2^-'  since  we  may  no  longer  have  mutually 
exclusive  sub-ranges.  If  either  A  or  F  does  not  satisfy  the  set  of  restrictions  listed 
above,  if  they  are  not  functions,  or  if  an  arbitrary  partition  H  is  used,  then  the  exten¬ 
sions  of  A,  P  and  N  are,  in  general,  not  well-deflned.  They  can,  however,  edways  be 
totally-deflned  by  adding  the  states  a^  =  (a.^)  and 

Whenever  we  do  not  need  to  distinguish  between  the  M  a-states  or  (O-states,  we  will 
use  a=(a,^x])=(a,n)  and  Q=(io,n).  Notice  that  we  have  implicitly  extended  the  notion 
of  a  message’s  staie  to  include  sets  of  states,  i.e.  loosely. 

{s  ,\^\i^I\):^\{s 

The  following  example  will  be  used  to  explain  the  model:  a  message  system  sup¬ 
ports  a  number  of  stations.  Two  in  particular  are  of  interest.  One  is  used  by  a  gradu¬ 
ate  student,  and  the  other  is  a  completely  automatic  station.  It  maintains  a  repository 
of  problems,  some  solved  and  some  unsolved.  The  student  may  ask  for  solved  or 
unsolved  problems  of  varying  degrees  of  difficulty  and  he  (or  she)  may  submit  solutions 
to  problems.  A  typical  message  from  the  student,  s  j,  might  be 

{GET, A  HARD  PROBLEM ,<prQbLem>,<soLution>). 

The  <probleTn>  and  <solutioTi>  field  would  be  blank  initially,  but  would  be  filled  in  by 
Sg  and  Sj  respectively. 

If  the  student  asks  for  a  simple  problem  and  solves  it,  he  may  ask  for  a  new  one.  If 
he  asks  for  a  difficult  or  unsolved  problem,  then  sg.  the  automatic  station,  indicates 
that  it  would  like  a  solution  submitted  by  changing  GET  to  SUBMIT.  If  he  finds  a  solu¬ 
tion,  then  A  HARD  PROBLEM  is  changed  to  A  SOLVED  PROBLEM  (automatically,  if  the 
system  can  check  the  correctness  of  solutions),  and  the  message  is  sent  back  to  Sg. 
When  Sg  receives  this  message,  it  saves  the  solution  to  the  problem  and  lets  the  mes¬ 
sage  die. 

The  control  attributes  are  the  first  two  fields,  since  these  are  the  only  ones  which 
affect  routing.  (We  assume  that  the  student  is  smart  and  always  finds  a  solution  to  any 
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problem,  so  <prQblerri>  and  <solutiQTL>  do  not  aflect  the  flow). 

For  our  purposes,  then, 

X'^Xi  xXg, 

r\^=\\GET, RETRIEVE,  •  •  •  [SAVE .STORE ,  •  •  •  jj 
Yl2=\\SOLVED  PROBLEM,  ■  •  •  \.\HARD  PROBLEM,  ■  -  H 
A  message  can  therefore  be  in  sight  states  depending  on  which  of  the  four  classes  it  is 
in  and  which  station  it  is  at.  There  are  two  more  states,  a  and  u,  corresponding  to 
message  creation  at  and  message  termination  at  sg. 


2. 1.  Rt  vectors 

"Whenever  the  partition  n  of  X  is  of  the  form  n=J~[ni  where  Hi  partitions  Xi,  we  can 
express  certain  collections  of  message  classes  by  bit  vectors.  Consider  a  message 
x  =  {xi,xz,  ,  .  .  ,Xn)-  Each  ni=|TT<i . Consequently  each  Zi  belongs  to  some 

We  therefore  have  The  bit  vector  b=(6j . 6^)  representing  x 

<=i 

is  obtained  by  setting  b^=2^^~^. 

For  example,  if  x  =  {GET,A  HARD  PROBLEM ,,),  then z  =7111X7722  and  b=(l,2). 

If  we  wish  to  consider  z^  belonging  to  any  of  several  rTy's,  then  we  add  their 

respective  bi's.  In  particular,  if  we  want  to  cdlow  ZierTy  for  any  je/iCU.S . Mij 

then  b^  therefore  ranges  from  0  to  2^-1.  If  each  bi  is  written  in  base  2,  so 

that  b  is  a  vector  of  strings  over  |0,lj,  then  we  can  easily  "read"  the  represented  blocks 
of  n.  In  the  above  example,  b=(01,10).  b=(10,ll)  would  represent  (I0.10)lj(10,01), 
and  so  on.  The  "on"  bits  stand  for  T7y ’s:  if  the  jth  bit  (reading  from  the  left)  of  b^  is  on, 
then  we  are  allowing  ZierTy .  The  total  set  of  "allowable"  message  classes  zell  is  the 

cross  pro dvx:t  of  the  rr^’s  represented  by  the  bit  strings,  i.e. 

t=i 

Note  that  if  each  6<=2^~\  eg.  b=(ll.ll).  then  all  the  bits  are  on.  and  z  is  allowed 
to  range  over  all  of  FI.  Conversely,  if  any  fai=0.  eg.  b=(  10,00),  then  the  cross  product  is 
empty,  end  no  zell  is  represented. 

A  message  flow  model  F  under  a  partition  n=f[ni  may  be  represented  by  a 

t=i 

directed  graph  D^(V,E)  where  7=5ij|a.«j  and  a  (directed)  edge  (s<.sy)  exists  for 
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every  non-empty  Py-.  Every  edge  is  labelled  by  a  set  of  bit  vectors  representing  all  the 
message  classes  which  are  allowed  to  flow  along  that  arc.  If  the  P^'s  consist  solely  of 
simple  conditions,  then  a  single  bit  vector  will  often  suffice  to  label  each  arc.  Yfe  also 
label  each  vertex  by  those  bit  vectors  representing  messages  that  may  undergo  a 
change  of  class.  Furthermore,  the  change  of  class  is  explicitly  indicated  by  marking 
with  an  asterisk  the  bit  in  corresponding  to  the  new  block  in  Hi  of  the  image  of  Xi  in 
A{s,x).  For  example,  only  submits  SOLVED  problems.  Mapping  SUBMIT  messages 
into  SUBMIT  SOLVED  messages  is  encoded  by  (10,11*),  le.  message  classes  (10,10) 
and  (10,01)  are  mapped  into  (10,01).  Vfe  call  the  labelled  graph  a  message  flow  graph 
(see  figure  1).  Note  that  not  every  bit  string  in  b  need  be  marked. 


Figure  1:  Amessage  flow  graph 

To  determine  what  paths  are  followed  by  all  message  classes,  we  "push"  the  bit 
vector  representing  all  of  IT  through  the  message  flow  graph.  When  the  bit  vector 
encounters  an  action,  it  "splits"  into  the  component  affected  and  the  one  which  is 
unaffected.  Furthermore,  the  affected  bit  vector  acquires  the  marking  of  the  action.  A 
marked  bit  vector  is  remarked  upon  encountering  an  action.  The  bit  strings  marked  in 
the  action  are  those  remarked  in  the  bit  vector  of  the  message.  In  this  way  we 
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maintain  both  the  original  message  class  and  the  current  message  class.  For  the  pur¬ 
pose  of  applying  the  action  and  routing  relations,  only  the  current  class  (or  set  of 
classes)  is  relevant. 

Suppose,  for  example,  that  the  set  of  messages  represented  by  the  bit  vector 
b=(ll^ll)  encounters  the  action  represented  by  (0*1,10).  b  started  its  life  as  (11,11) 
but  LS  currently  (01,11).  Only  the  current  value  is  of  Importance  in  evaluating  the 
effect  of  the  action,  but  we  must  also  keep  track  of  the  original  value.  The  action 
causes  b  to  be  split  into  (11  ,10)  and  (11*. 01),  i.e.  the  component  affected  by  the 
action  and  the  one  unaffected.  Applying  the  action  causes  the  bit  vectors  to  be 
remarked,  and  we  obtain  (1  *1,10)  and  (11  *,01). 

Bit  vectors  are  also  split  according  to  routing.  When  the  (current)  message 
classes  represented  by  a  bit  vector  must  follow  different  arcs,  that  bit  vector  must  be 
split  into  the  appropriate  components.  There  may  be  several  ways  to  do  this:  Suppose 
(11,11)  is  routed  in  two  directions.  If  one  of  the  arcs  is  labelled  (10,01)  and  all  other 
message  classes  follow  the  other  arc,  then  the  second  component  of  (ll.ll)  can  be 
represented  by  (ll,10)tj(01.0l)  or  (I0,10)u(01.1l)  or  (10, 10)u(01.10)u(01.01).  For 
illustration  see  figure  1  and  for  a  more  detailed  account  see  [TsicBl]. 

2.2.  Non-determinism  in  the  MTM 

Although  A  and  P  are  defined  as  relations,  we  have,  up  till  now,  more  or  less 
assumed  them  to  be  functions.  We  have  edso  placed  some  ostensibly  unreasonable  res¬ 
trictions  on  the  nature  of  the  actions  and  routing  predicates.  It  is  unlikely,  in  practice, 
to  only  have  actions  that  set  attributes  to  constants.  It  is  also  unreasonable  to  expect 
that  simple  conditions  will  suffice  to  express  all  desired  routing  predicates.  Finally,  it 
is  possible  that  A  and  P  depend  on  some  external  information  (such  as  user  input), 
thus  rendering  them  non-deterministic  within  the  framework  of  the  message  flow 
model. 

It  is  clear  that  this  case  prevents  A  or  P  from  being  well-defined.  Furthermore,  if 
actions  do  not  set  attributes  to  constants,  or  if  arbitrary  conditions  are  used  in  routing 
predicates,  then  A  and  P  may  be  well-defined  over  5xA",  but  not  always  over  5x11.  An 
arbitrary  action  may  map  one  message  class  into  any  number  of  other  message 
classes.  £sxn  induced  by  the  partition  5x11,  is  necessarily  an  equivalence  relation,  but 
may  not  be  a  congruence  relation  for  N.  By  the  same  token,  arbitrary  routing 
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functions  applied  to  a  state  (s  ,x)  may  not  map  neatly  to  a  single  station  for  each  xE£. 
It  may  be  possible  to  choose  H  so  that  A.  and  P  are  well-defined  over  states,  but,  in  the 
worst  case,  n=||x  j  [xeXj.  We  must  therefore  be  prepared  to  handle  non-determinism 
If  we  expect  to  obtain  any  worthwhile  reduction  of  message  classes  in  some  situations. 

A  non-deterministic  message  flow  graph  differs  from  its  deterministic  counterpart 
only  in  that  the  bit  vectors  labelling  out-edges  from  a  station  (i.e.  routing  predicates) 
may  not  be  mutually  exclusive,  and  that  the  bit-vectors  labelling  stations  (i.e.  actions) 
may  have  several  marked  bits  in.  a  given  bit  string.  The  multiple  marking  of  an  action’s 
bit  vector  Indicates  the  image  of  that  action  when  applied  to  messages  represented  by 
the  bit  vector. 

The  non-determinism  Introduced  into  the  model  now  prevents  each  message  class 
from  being  associated  with  a  particular  path.  The  collection  of  possible  message  paths 
taken  by  messages  in  a  block  of  IT  can,  however,  be  expressed  by  regidar  expressions 
[Ginz68].  We  first  extend  the  notion  of  a  message's  history  $(s,x)  over  {SxX)*  to  mes¬ 
sage  states.  We  have,  therefore,  'i>(s,x)=(s,x)$(jV(s,x))  yielding  a  string  or  a  choice  of 
strings  in  (5x11)^. 

Theorem  1:  Message  histories  in  a  non-deterministic  message  flow  model 
F=<S ,X,A.P>  can  be  expressed  by  regular  expressions. 

Proof:  Let  H  be  a  partition  of  X.  Consider  the  finite  automaton  A=<  T,  V",^,a,f(yj>.  The 
states  V  of  the  automaton  are  the  states  of  the  message  flow  model,  Sxnula.oj. 
together  with  the  special  a-  and  cj>-states  The  inputs  are  also  V'.  The  next-state  relation 
^{Vi.Vz)  holds 

(1)  if  i;i=a  and  vg  is  initial 

(ii)  if  N{vi)=V2 

(iii)  if  1^1  is  final  and  vz-o. 

Note  that  A^(si,xi)  =  (s2,X2)  if3xiexi,  X2€.r2  .3’7V(si,Xi)=(s2,X2).  Neither  N  nor  ^  need 
be  well-defined.  Since  the  input  strings  accepted  by  A  are  precisely  the  valid  histories 
in  of  message  classes  In  0,  those  histories  may  be  expressed  by  regular  expressions 
□  . 

Corollary  2:  Message  paths  in  F  may  be  expressed  by  regular  expressions. 

Proof:  The  message  paths  are  obtained  from  the  regular  expressions  In  theorem  1  by 
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the  mapping  1^1(5, x)=s  extended  to 
maton.  A'=<y’,5'',(‘',a,cj>,  where  ^’( 
strings  accepted  by  A'  are  the  valid 
are  therefore  regular  expressions  □ . 


strings  in  (5’xn)*.  Alternatively,  consider  the  auto- 
holds  if  3yz  =  {s2,x^.^^{vi,v2)  holds.  The 
message  paths  taken  by  message  classes  in  11,  and 


The  transition  graph  [Ginz60]  of  A  is  the  directed  graph  D'{V,E')  where 

may  be  thought  of  as  an  inefficient  representation  of  the  mes¬ 
sage  flow  graph,  D.  We  call  D'  the  state-graph  of  F.  (See  figure  2.) 


a(ll.ll) 


f 


cj(lO.Ol) 


(unattainable) 


Figure  2:  A  state  graph 

Any  state  not  reachable  from  a  is  unattainable.  Any  state  from  which  o  is  reach¬ 
able  is  terminating.  If  D'  has  no  unattainable  states,  then  it  has  exactly  one  vertex  a 
with  zero  indegree  and  one  vertex  cj  with  zero  outdegree.  If  F  is  a  deterministic  mes¬ 
sage  flow  model  with  respect  to  FI,  then  every  vertex  i;  eV’N  [a,cjj=5xn  has  outdegree 

Two  vertices  in  a  directed  graph  are  diconnected  if  each  is  reachable  from  the 
other.  Diconnection  is  an  equivalence  relation  whose  resultant  partition  induces  subdi¬ 
graphs  called  dicomponents  [BoMu76].  If  F  is  deterministic,  then  the  non-trivlal 
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dicomponents  of  D'  eire  cycles:  if  a  message  encounters  a  state  twice  in  its  history,  it 
must,  ipso  facto,  cycle  indefinitely.  (Not  so  if  F  is  non-deterministic.)  We  have,  there¬ 
fore.  the  following  classification; 

Proposition  3;  If  a  message  flow  model  F  is  deterministic  with  respect  to  FI,  then  the 
history  of  an  initial  message  class  is  either  or 

■  ■  ■  ■^0'- 

Furthermore,  the  state  graph  D'  of  F  has  the  property  that  the  connected  subgraphs 
of  jC>'[5xn]  are  either 

(i)  directed  trees  wth  a  single  final  state  as  a  root  node;  the  root  node  is 
reachable  from  every  vertex;  there  may  be  dead  branches  of  unattainable 
states;  all  attainable  states  are  terminating 

(ti)  directed  graphs  with  a  single  directed  cycle;  the  cycle  is  reachable  from 
every  vertex;  there  may  be  dead  branches;  all  states  are  non-terminating  □. 

We  may  describe  non-deterministic  message  flow  models  by  considering  dicom¬ 
ponents; 

Proposition  4:  Let  D'  be  the  state  graph  of  a  message  flow  model  F  with  respect  to  U. 
The  condeTisatian,  C(V’),  of  D’,  obtained  by  collapsing  dicomponents  into  vertices,  is 
characterized  as  follows: 

(1)  attednable.  terminating  dicomponents  form  a  lattice  under  meet  and  join 
induced  by  reachability 

(ii)  unattainable  dicomponents  may  lead  into  the  lattice 

(iii)  critical  dicomponents  may  lead  out  of  the  lattice  to  non- terminating 
dicomponents  □. 

One  immediate  consequence  of  this  is  that  once  a  message  has  left  a  dicomponent 
it  may  never  return  to  it.  Since  a  condensation,  or  dicomponent-graph,  is  acyclic,  a 
message  may  never  return  to  a  state  once  it  has  entered  a  new  dicomponent.  This  may 
lead  us  to  regular  expression  for  message  paths  which  are  simply  the  concatenations  of 
the  regular  expressions  for  dicomponents. 

We  may  use  classical  techniques  for  deriving  the  regular  expressions,  but  we  may 
save  some  effort  by  using  D.  the  message  flow  graph,  instead  of  D’,  the  state-graph,  as 
our  representation  of  F  and  of  our  finite  automaton.  Regular  expressions  are  obtained 
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by  solving  T  =  BT  +  E  where  T  and  E  are  column  vectors  and  5*  is  a  matrix,  Each  is 
the  regular  expression  we  seek  for  state  i  of  the  automaton.  E^  is  A  (the  empty  string) 
if  1  is  a  final  state  and  0  otherwise.  The  entries  are  the  inputs  required  to  advance 
from  state  i  to  state  The  solution  to  T  is  B*E,  where  B*=I -^-B i-B^+B^  •  •  ■  ,  but  this 
format  is  useless  to  us  for  determining  the  individual  Ti's.  Regular  expression  are 
non-commutative,  so  conventional  matrix  equation  solving  techniques  are  also  useless 
here.  (They  are  useful,  however,  for  solving  the  generating  functions  for  each  regular 
expression,  since  those  do  commute.)  The  laborious  method  of  substitution  and  elimi¬ 
nation  is  therefore  used  to  solve  these  equations. 

We  may  save  some  effort,  however,  if  we  conglomerate  message  states  using  bit 
vectors.  Instead  of  deriving  regular  expressions  for  each  (s,^)  in  5x11,  we  attempt  to 
do  30  for  (s,b)  where  b  Lnitially  represents  ail  of  FI,  and  is  split  only  when  necessary.  It 
is  not  possible  to  write  a  matrix  equation  for  this,  since  we  do  not  initially  know  how  the 
bit  vectors  will  be  split.  We  therefore  do  not  know  the  size  of  T  until  we  have  already 
solved  it. 

We  let  hh  represent  the  regular  expression  for  (A:,b).  We  wish  to  represent  mes¬ 
sage  paths  so  our  Inputs  are  stations.  Our  input  in  state  {k  ,h)  is  k.  If  the  next  station 

is  any  one  of  Atj . kt,  then  we  write  A:b=A:A:jbi+  •  •  •  +A:A:^b{.  If  A:  is  a  final  station. 

then  kf  is  cj.  If  f>l  then  there  is  a  possibility  of  termination  or  continuation.  If  t  =  l, 
then  all  messages  terminate.  Each  bj  is  the  remarked  component  of  b  which  is  routed 
to  ki.  U  F  is  non-deterministic,  then  the  bj’s  need  not  be  mutually  exclusive. 

Consider  the  previous  example  with  stations  \a,l,Z,o\  and  n=(ll,ll):  >li  =  (10.11*), 
/42=(0*1.10).  i^i2=(ll.ll).  /'2i  =  (11.10)U(0I.0l)  and  P2«  =  (10.0l)'  Messages 

enter  the  system  at  station  1.  get  passed  to  station  2  and  back,  and  possibly  leave  the 
system  at  station  Z.  Messages  may  be  modified  at  either  station  depending  on  their 
contents. 

We  write: 

a(ll,ll)=al(ll.ll) 

1(11.11)  =  1(10,11)+1(01.11) 

=  I2(10,11*)  +  12(01.11) 

2(l0.0l)  =  2cj(l0,0l) 

2(01.1l)=2(01,10)-t2(01.0l) 
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=  21(0'1,I0)  +  21(01,01) 

1(10.I0)  =  12(10.10') 

=  12cj(l0.10*^ 

l(0l,0l)  =  12(01,0l) 

=  (12)"(0I.01) 

Substituting  back  we  ootain: 
a(ll.ll)=al2cj(l0.11*) 

-Hal212a)(0*1.10') 

-»-a(12)”(01,0l). 

We  may  now  "read"  the  message  paths  as  being  the  strings  preceding  the  message 
classes  in  the  above  expression.  In  this  example  there  are  two  terminating  message 
paths.  All  terminating  messages  end  in  the  class  (10,01).  Only  messages  In  the  class 
(01,01)  do  not  terminate,  (Note  that  (2,(10,10))  is  an  unattainable  state  and,  as  such, 
2(10,10)  appears  nowhere  in  the  above  equations.  2(10,11*)  is  really  2(10,01)  since  we 
are  interested  only  in  current  states.)  The  only  non-trivial  dicomponent  is 
[  1(01, 01), 2(01,01) j.  A  terminating  dicomponent  would  make  a  contribution  to  the  regu¬ 
lar  expression  of  the  form  {expression)*.  We  have  saved  a  small  amount  of  work  by 
using  bit  vectors,  since  we  would  have  otherwise  used  16  equations  for  4  stations  times 
4  message  clcisses. 

The  next  example  illustrates  how  non-determinism  is  handled  by  the  model;  we 
have  stations  \a,l,Z,u\  and  n  =  (ll,ll)  as  before.  Furthermore,  Ai=(l0, 1  *0*), 
A2=(11.11).  Fiu=(01.10),  F2i=(11.10)U(01.01)  and 

F2o=(  10,01)  (see  figure  3).  We  have  introduced  two  kinds  of  non-determinism:  either 

depends  on  some  external  parameter,  or  our  partition  H  is  not  fine  enough  to  give  us 
congruence  classes  over  X^.  In  our  example  the  student  is  allowed  to  submit  "solu¬ 
tions”  to  problems  which  he  decides  to  classify  as  SOLVED  or  UNSOLVED .  In  the  latter 
case  he  may  be  submitting  a  tentative  solution  and  asking  for  a  hint.  We  have  a  similar 
situation  with  Piz  and  Pxu-  Since  they  are  not  disjoint,  our  routing  is  non- 
deterministic:  The  student  now  has  a  choice  of  requesting  a  fresh  HARD  PROBLEM  or 
giving  up.  S2  no  longer  insists  on  getting  a  solution.  Nevertheless  we  may  proceed  as 
before: 


c((ll,ll)=al(ll,ll) 
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Figure  3:  A  non-deterrainistic  message  JXq'w  graph 

1(1 1. 1 1)  =  1(10. 10)  + 1(01. 10)  +  1(  11,01) 

=  12(l0,r0')+li.;(01.10)  +  12(01. 10)  + 12(1 1.01) 
2(l0.1l)=21(l0.l0)+2cj(l0.0l) 

=212(10, r0*)+2cj(:0. 01) 

=2(12)Vl0.1l') 

2(01.10)  =  21(01,10) 

=212(01, 10)  +  21cj(01. 10) 

=2l(2l)'cj(01.10) 

2(ll,0l)=2cj(l0.0l)  +  2l(01.0l) 

1(01.01)  =  12(01.01) 

=  (12)“(01.01) 

Substituting  back  we  obtain: 

a(ll.ll)=al2(l2)*cj(10.10') 

+alii>(01, 10) 

+al2l(2l)'cj(01.10) 

+al2cj(l0.0l) 
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4-a(12)-(01,0l). 

Notice  that  the  second  and  third  terms  simplify  to  al(2l) ^u;(01,10).  As  before  we  can 
read  off  the  terminating  message  paths;  al2(l2)^c>;,  al(2l)*ii>  and  al2c^.  The  new 
dicomponents  are  |l(l0,l0).2(l0,10)j  and  ^1(01, 10), 2(01, 10)|.  There  are  no  unattain¬ 
able  states.  Messages  that  terminate  end  up  in  classes  (10,01)  or  (01,10).  Messages  in 
class  (10,01)  are  guaranteed  to  terminate.  Messages  in  class  (01,01)  will  never  ter¬ 
minate.  Message  classes  (10,10)  and  (01,10)  may  or  may  not  terminate  depending  on 
the  true  nature  of  Ai  and  Pjy  (see  figure  4). 

If  a  finer  partition  H  would  eliminate  the  non-determinism  then  those  paths  will 
also  terminate.  Similarly,  if  Ai  and  depend  on  some  external  variable,  then  we 
most  show  that  a  provably  finite  number  of  iterations  will  enable  our  messages  to  exit 
their  loops.  Finally,  if  Ai  and  Py^  are  "genuinely  non-deterministic”  (le.  random),  then 
we  have  only  a  probablistic  guarantee  of  termination. 


a(ll.ll) 


Plgure  4:  ATion-deterrainistic  state  graph 
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3.  Concluding  remarks 

The  message  how  model  presented  in  this  paper  provides  us  with  a  way  to  charac¬ 
terize  tasks  in  a  message  management  system.  The  regular  expressions  which 
describe  the  paths  taken  by  classes  of  messages  may  be  used  in  the  analysis  of  a  given 
system  for  correct  behaviour.  We  will  continue  to  extend  and  develop  the  model.  Coor¬ 
dination  of  messages,  for  example,  is  not  yet  represented  The  power  of  hierarchical 
decomposition  of  stations  Into  substations  corresponding  to  tasks  and  subtasks  has  not 
been  explored.  We  also  hope  to  incorporate  the  model  within  a  working  message 
management  system  capable  of  specification  of  automatic  message  handling  pro¬ 
cedures  [TRGHBl]. 

The  area  of  regular  expressions  also  suggests  many  possible  applications  of  the 
model,  in  particular,  the  assignment  of  weights  to  stations  will  yield  generation  func¬ 
tions  for  costs  associated  with  classes  of  messages.  The  amount  of  work  taken  to  pro¬ 
cess  a  particular  message,  or  the  probability  of  arriving  at  a  particular  state  may  be 
determined  using  generating  functions.  This,  in  turn,  may  suggest  ways  of  improving 
performance  by  redistributing  workloads  or  restructuring  the  system. 
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ABSTRACT 

We  describe  a  message  filing  capability  which  allows  for  the 
retrieval  of  messages  according  to  contents.  Messages  are  organ¬ 
ized  in  large  general  files  such  that  frequent  reorganization  is 
avoided.  The  user  specifies  a  filter  which  retstricts  the  attention 
to  a  msinageable  subset  of  messages.  Messages  within  the  subset 
are  retrieved  for  a  fined  check.  We  discuss  file  organization  and 
access  method,  as  well  as  performance  and  implementation  con¬ 
siderations. 


1.  Introduction 

Consider  an  office  information  system  and  the  messages  which  circulate 

within  it.  We  will  assume  for  the  purposes  of  this  paper  that  the  messages  are 

*  In  Proc.  ACM  SIGOA  Conf.  on  Office  Information  Systems, 
Philadelphia,  PA,  June  1982. 
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text  messages  in  machine  readable  form,  ViTe  are  Interested  in  designing  a  mes¬ 
sage  filing  capability  ■which  has  the  follo'wing  characteristics: 

1)  It  can  deal  ■w'ith  a  wide  variety  of  messages 

2)  It  can  retrieve  the  messages  in  a  flexible  manner 

3)  It  provides  a  simple  and  uniform  interface  to  the  user 

4)  It  can  be  implemented  efin.ciently  for  a  large  volume  of  messages 

We  deflne  messages  as  consisting  of  a  /leader  and  a  body  [Kirstein],  The 
header  contains  formatted  data  representing  the  most  important  characteris¬ 
tics  of  the  messages,  e,g.,  sender,  date,  destination,  etc.  The  body  is  text  con¬ 
sisting  of  a  series  of  words.  We  will  denote  by  Aq,Ai,  •  •  •  t.he  attributes  of  the 
header.  Aq  is  a  special  attribute  which  contains  a  unique  system-wide  identifier 
for  the  message.  The  body  wiU  be  denoted  as  an  attribute  B  of  type  text.  A  par¬ 
ticular  message  will  be  represented  as  (aQ.aj.  •  •  •  All  messages  do  not 

have  to  be  of  the  same  type.  Each  type  of  message,  however,  is  represented  by 
a  set  of  attributes  and  a  body.  In  the  two  extreme  cases  we  have  the  message 
type  {Aq,B),  Le.,  documents  and  {Aq,Ai,  •  •  ■  ,An),  i.e.,  records.  Forms  as  mes¬ 
sages  can  be  represented  as  (Aq.  ’  ’  '  tAji.B)  "with  the  additional  stipulation  that 
the  values  of  A’s  are  dispersed  within  B. 

2.  Mess2tge  storage  and  retrieval 

People  in  offices  find  paper  messages  by  filing  them  appropriately.  A  nies- 
sags  file  is  a  labelled  container  of  messages  which  groups  messages  according  to 
their  meaning.  For  instance,  messages  from  a  particular  person  about  a  partic¬ 
ular  project  are  filed  together.  Notice  that  a  message  file  in  the  office  environ¬ 
ment  allows  filing  of  different  types  of  messages  within  it  (unlike  a  computer 
file).  To  help  pinpointing  the  right  file  name  for  retrieval,  file  names  can  be 
structured  according  to  their  relationships.  A  common  structure  is  a  hierarchi- 
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cal  structure  as  provided,  for  instance,  by  folders  within  drawers  of  fUe  cabinets. 
People  search  a  message  file  by  scanning  sequentially  to  obtain  the  appropriate 
message(s).  They  are  guided  by  a  partial  specification  of  contents  of  the  mes¬ 
sage  and  are  assisted  by  a  certain  vague  image  of  what  the  message  looks  like. 

There  are  some  characteristics  of  office  message  files  which  we  would  like 
to  retain  for  eiectronic  filing.  However,  there  are  also  limitations  which  we  can 
and  should  avoid.  First,  we  observe  that  it  would  not  be  difficult  to  duplicate  the 
same  kind  of  manual  message  filing  in  terms  of  electronic  messages.  For  exam¬ 
ple,  UNIX  files  provide  hierarchical  directories  for  file  names  [Ritchie  and 
Thompson].  Within  each  file  messages  can  be  stored  sequentially  as  byte 
strings.  We  only  need  to  implement  a  sequential  scan  of  messages  which  takes  a 
sequence  of  bytes  and  displays  it  as  a  message.  We  would  expect  to  find  such 
capabilities  associated  with  any  kind  of  file  server  as  part  of  an  office  informa¬ 
tion  system.  However,  there  are  some  limitations  in  such  a  filing  facility. 

First,  the  structure  of  the  files  is  seldom  static.  The  world  changes,  the 
message  characteristics  change  and  their  filing  method  should  change  accord¬ 
ingly.  Even  if  we  have  achieved  a  perfect  filing  method  encapsulated  in  a  direc¬ 
tory  structure,  it  will  soon  be  out  of  date.  Reorganization  of  the  files  is  difficult 
even  in  a  computer  environment.  Messages  will  have  to  be  reassigned  to 
different  files  in  a  new  directory.  Second,  many  messages  can  potentially  refer 
to  many  subjects  and  should  be  associated  with  many  files.  In  such  a  case,  we 
either  have  to  duplicate  the  messages  (with  associated  consistency  problems)  or 
we  need  to  have  pointers  to  messages  rather  than  the  messages  in  many  files. 
Finally,  no  matter  what  file  organization  we  choose  we  will  not  be  able  to  divide 
easily  messages  in  small  groups  according  to  contents.  As  a  result,  if  the 
volume  of  the  messages  becomes  high,  the  files  will  be  large  and  sequential  scan 
will  be  time  consuming. 
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We  should  augment  the  filing  capability  >vith  an  access  method  "which  cam 
retrieve  messages  according  to  contents.  In  this  way  messages  can  be  organ¬ 
ized  in  general  files  rather  than  complex  directories,  e.g.,  1981  general 
correspohdehce,  project  X  memos,  etc.  The  information  retrieval  capability 
enables  the  user  to  retrieve  quickly  the  desired  m.essage  from,  the  file.  Since 
the  messages  are  filed  in  very  general  terms,  frequent  reorganization  of  files  is 
also  not  needed. 

It  would  be  nice  to  retain  the  properties  of  sequential  scan.  That  is,  mes¬ 
sages  are  still  retrieved  one  at  a  time  simulating  a  sequential  scam.  It  is,  how¬ 
ever,  a  ’’lucky"  sequential  scan.  The  user  specifies  a  fUter.  Messages  which  do 
not  qualify  according  to  the  filter  are  skipped.  Messages  which  qualify  are 
displayed.  It  is  also  important  that  they  are  displayed  in  a  format  recognizable 
to  the  user.  In  this  way,  the  user  can  visually  check  the  message  before  he 
accepts  it. 

In  this  paper  we  will  concentrate  on  the  specification,  analysis,  and  imple¬ 
mentation  of  such  a  filtering  capability.  We  will  assume  that  all  messages  are 
stored  in  large  general  files  according  to  user  and  role.  The  specified  filter  res¬ 
tricts  the  attention  to  a  manageable  subset  of  the  messages.  Messages  "within 
the  subset  are  obtadned.  sequentially  for  a  final  check  by  the  user. 

A  major  thesis  of  the  paper  is  that  the  filtering  capability  does  not  have  to 
be  exact.  We  assume  that  the  user  will  seldom  be  able  to  specify  an  absolutely 
tight  filter.  His  specification  in  terms  of  contents  will  allow  more  messages  to 
qualify  in  addition  to  the  ones  he  absolutely  wants.  His  visual  inspection  will 
finally  pinpoint  the  desired  messages.  If  the  specification  of  the  filter  is  not 
exact,  its  implementation  does  not  have  to  be  exact.  In  other  words,  suppose 
the  specification  of  the  filter  allows  10%  non-relevant  messages.  If  the  imple¬ 
mentation  of  the  filter  adds  another  0.5%  of  non-relevant  messages,  the  user  will 
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not  care.  The  system  could  eliminate  this  extra  0.5%  with  some  additional  pro¬ 
cessing.  However,  it  is  as  easy  for  the  user  to  deal  with  10.50%  as  to  deal  with 
10%  of  unwanted  messages.  This  resiliency  is  present  because  we  pass  the 
results  of  the  query  to  the  user  and  not  to  a  program. 

3.  File  orgaiiization 

Given  a  specification  of  the  filter  in  suitable  internal  form  we  need  a  file 
organization  which  restricts  our  attention  to  the  set  of  appropriate  messages. 

The  specification  of  values  for  ^l^’s  can  medce  use  of  standard  indexes,  e.g., 
B-trees  for  their  evaluation.  The  specification  and  evaluation  of  a  pattern  is 
more  difficult.  We  could  use  indexing  methods  for  words  but  we  will  need  much 
space  to  store  the  word  indexes  [Haskin].  In  addition  merging  the  pointers 
obtained  from  the  indexes  is  a  time  consuming  problem  [Haskin].  We  propose 
instead  a  method  of  reducing  the  sequential  search  to  a  manageable  level.  The 
organization  of  the  messages  is  sequential.  The  search  is  still  sequential  but  we 
search  an  auxiliary  file  rather  than  the  message  file. 

Consider  a  word  W  appearing  in  a  message.  We  will  denote  by  S{W)  a  bit 
string  which  represents  (not  uniquely)  the  word  W.  There  are  many  different 
algorithms  for  transforming  W  into  S{W).  One  method  is  to  partition  W  into  tri¬ 
plets  or  quadruplets  of  letters  and  to  hash  each  partition  of  letters  into  a  fixed 
number  of  bits.  The  bits  are  concatenated  (up  to  a  maximum  length)  to  form 
the  signature  S{W).  A  fixed  number  of  bits  may  preceed  the  signature  5(fy)  to 
indicate  its  length.  Alternatively,  signatures  of  words  may  be  of  fixed  length. 

Consider  a  set  of  messages  of  the  form  77i.=(aQ.ai,  ’  '  ’  <^)-  Suppose  that  we 
physically  file  all  the  messages  sequentially  as  they  come  into  a  general  physical 
file  F.  Let  be  a  pointer  to  a  message  within  F.  That  is  the  file  system  will 
deliver  the  message  on  the  basis  of  y..  We  will  also  construct  signature  files 


101 

■which  will  aid  the  search  for  the  message.  > 

When  the  user  stores  a  message  he  ■will  specify  one  (or  more)  logical  files  in 
which  he  wants  the  message  filed.  For  each  logical  file  f  there  ■vfill  be  an  associ¬ 
ated  physical  file  f  g  which  stores  not  the  message  but  a  representation  of  the 
message  which  we  will  call  the  signatuTe  of  the  message.  The  signature  of  the 
message  will  consist  of  ,5(0^  ).  •  •  •  S{a^),S{b  ))  where: 

i*  is  a  pointer  to  where  the  message  is  stored 
t  is  the  type  of  the  message 

S{a^ ).  •  •  ’  ,S(a^)  are  exact  representations  of  a, ,  •  •  •  ,a„ 

S{b)  is  the  signature  of  the  body. 

The  signature  of  the  body  is  a  sequence  of  bits  -which  approximately  represents 
the  significant  words  of  the  message’s  body.  This  is  obtadned  by  the  following 
algorithm.  Traverse  sequentially  the  body  b.  For  each  word  check  whether  it  is 
one  of  a  set  of  common  words  (the,  that,  a,  ...).  If  it  is  set  S{Wi)  -  (p,  i.e.,  skip 
the  word.  If  it  is  not  substitute  the  word  W  with  its  signature  S{W).  Skip  also 
the  blanks  and  formatting  symbols. 

The  sequence  of  words  comprising  h  is  represented  by  the  bit  string  5(b )  in 
the  following  manner. 

If  a  word  jy  is  in  b  then  S{W)  -will  be  a  substring  of  S{b)  pro-vdded  that  W  is 
not  a  common  word.  Notice  that  the  opposite  is  not  true.  That  is,  5'(fy)  may 
match  a  substring  oi  S  {b)  without  W  be  a  part  of  b .  This  situation  may  arise  for 
two  reasons.  First,  two  words  W  and  W  may  have  the  same  signature 
S{W)  =  S{W').  We  may.  therefore,  find  W  rather  than  IK  in  b .  Second,  if  we  use 
variable  length  word  signatures  ■without  a  prefix  that  specifies  the  length  of  the 
signature,  the  bitstring  S{W)  may  match  the  suffix  and  prefix  of  the  signature  of 
words  W  W”.  This  situation  arises  because  blanks  are  eliminated. 
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Consider  a  series  of  messages  [rrii,  ■  ■  'vvith  bodies  ^6^,  •  •  •  and  a 

'^ord  The  set  matches  is  a  superset  of  the  set  \7nj  \  W 

matches  6j  j.  It  is  important  that  the  difference  between  the  cardinality  of  the 
two  sets  is  small.  We  will  come  back  to  this  point. 

4.  Access  method 

A  user  specifies  a  query  to  the  message  filing  facility  by  first  indicating  the 
logical  file  /  and  the  type  of  message  t  he  is  accessing.  The  system  displays  the 
appropriate  template  for  f.  The  user  optionadly  specifies  some  vEilues  for 
A^,Ai,  •  •  •  ,An  and  a  pattern  P  by  partially  filling  the  template.  Values  entered 
for  Aq,Ai,  •  ■  ■  An  are  interpreted  as  selection  conditions.  The  pattern  P  can  be  a 
complicated  expression  of  words  involving  arbitrary  boolean  combinations  of 
regular  expressions  of  words.  Messages  that  satisfy  the  query  are  these  that 
satisfy  the  conjunction  of  all  selections  specified,  and  the  pattern  P  of  words 
appears  in  the  body  of  the  message. 

To  find  the  set  of  qualifying  messages  the  system  scans  sequentially  the  sig¬ 
nature  file  of  S  associated  with  f  .  For  each  message  signature  it  checks  first 
the  type.  If  the  type  does  not  match  it  skips  the  signature.  If  it  does  it  tries  to 
match  first  the  given  values  for  the  attributes  and  then  the  signature  of  the 
body. 

Consider  the  pattern  P  as  expressed  by  a  regular  expression  of  words.  The 
signature  of  the  pattern  S{P)  is  a  regular  expression  which  is  obtained  by  sub¬ 
stituting  S{W)  for  each  significant  word  in  P  retaining  the  regular  expression 
operators,  dropping  common  words  and  substituting  for  each  uncon¬ 

strained  interval  of  words.  Matching  message  bodies  for  P  is  substituted  with 
matching  the  signature  of  P,  S (P)  against  the  signature  of  the  bodies  S{b).  The 
signatures  S(b)  are  binary  strings  and  S{P)  is  a  regular  expression  of  binary 
strings.  Testing  5(6)  for  S{P)  is  equivalent  with  testing  whether 
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S{b)e  Puir  S{F)10\jU* 

This  is  a  recognition  problem  for  firlte  automata  [Aho,  Hopcroft,  Ullman].  It 
can  be  sho'wn  that  for  a  specific  regular  expression  R  the  membership  of  a 
string  in  R  can  be  checked  by  a  NFA  with  number  of  states  at  most  double  the 
number  of  symbols  In  R  [Aho,  Hopcroft,  Ullman].  It  follows  that  given  a  pattern 
P  testing  for  S{P)  in  the  bit  strings  S{b)  can  be  done  by  a  NFA  with  states  a 
linear  function  of  the  symbols  in  the  pattern  P. 


5.  Performajice  considerations 

The  performance  of  the  file  organization  described  depends  on  the  percen¬ 
tage  of  unwanted  messages  retrieved,  the  relative  cost  of  the  sequential  scan  of 
a  block  of  the  signature  file  versus  the  cost  of  accessing  a  block  of  text,  and  the 
relative  sizes  of  the  signature  file  and  the  text  files.  The  messages  that  qualify 
by  testing  the  signature  S{b)  for  matching  with  the  signature  of  a  pattern  P, 
5(P).  is  a  superset  of  messages  that  qualify  for  P.  Larger  signatures  of  words 
will  result  in  less  collisions  (because  the  probability  that  two  different  words 
hash  into  the  seime  signature  becomes  smaller).  However,  larger  signatures 
imply  extra  cost  for  sequentially  scanning  the  signature  file.  Thus,  there  is  a 
tradeoff  between  the  sequential  scan  of  a  long  signature  file  and  the  retrieval  of 
a  large  superset  of  blocks  from  the  message  file. 

In  the  following  we  will  assume  that  signatures  of  words  are  of  fixed  length 
of  L  bits.  This  eliminates  the  need  for  a  prefix  in  the  signature  to  specify  its 
length,  and  it  does  not  favor  long  words.  Similar  analysis  can  be  applied  for 
non-flxed  length  signatures  by  using  statistics  on  the  lengths  of  words  in  printed 
English  [Bourne  and  Ford].  Let  D  be  the  number  of  distinct  words  in  the  text, 
and  C  the  number  of  possible  signatures.  For  L  bit  word  signatures  C  —  2.^. 
Given  a  word  W  with  signature  S{W)  we  want  to  know  what  is  the  expected 
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number  of  other  words  W  such  that  5(?f)  =  S{W).  This  can  be  modelled  as  a 
selection  with  replacement  problem:  For  each  distinct  word  V*  we  select  ran¬ 
domly  a  signature  S(W).  The  probability  that  k  distinct  words  are  mapped  into 
a  given  signature  is: 


(k)  = 


(1) 


c'  C' 

Thus  the  probability  that  more  than  one  word  W  has  the  same  signature 
•S’!  )  =  >?(^)  with  a  given  word  fK  is  given  by: 


D 

J]Pk 

j3  {collision)  -  — 

-  -  -  EPfc 

k  =  \ 


1  -  p(0)  -p{l) 

1  -p(0) 


The  number  of  distinct  words  can  be  directly  found  from  the  text.  Thus,  for 
given  C  and  D  the  probability  that  more  than  one  words  have  the  same  signa¬ 
ture  can  be  estimated.  One  would  think  that  for  large  text,  D  would  be  very 
large.  However,  words  in  printed  English  follow  Ziph’s  law  [Ziph].  Shannon  has 


used  the  formula  —  to  relate  word  frequency  to  rank,  and  estimated 

n 

D  =  8727  [Shannon].  A  more  detailed  estimation  [Grignetti]  gave  D  =12370,  and 
Dewey’s  tables  draTm  over  a  sample  of  more  than  100000  words  found  10119  dis¬ 
tinct  words  [Dewey].  In  comparison  Ci  =  2®  =  256  for  8  bit  signatures  and 
C2  -  65536  for  16  bit  signatures. 


Let  I  be  the  average  occurences  of  a  distinct  non-trivial  word  in  the  text 
file.  To  the  k  distinct  words  that  have  the  same  signature  corresponds  a  set  of 
kl  non-trivial  words  of  the  text  file.  These  words  are  placed  into  a  number  of 
B{kL  ,N ,M)  blocks  of  text,  where  N  is  the  total  number  of  non-trivial  words,  and 
M  is  the  total  number  of  blocks  of  the  text  file.  Given  k,  I,  N  and  M,  B{kl ,N ,M) 
can  be  estimated  as  the  result  of  an  experiment  of  selecting  without 
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replacement  kl  balls  from  a  pool  of  N  bedls  containing  balls  of  M  different  colors 
([Cardenas],  [Yao],  [Christodoulakis  a],  [Christodoulakis  bj).  A  closed  form  for¬ 
mula  for  randomly  placed  words  among  the  blocks  of  the  file  is  [Yao] 


B{kl,N,M)  =  M{1  -  Yli 


r=0 


N  —  T 


-)) 


Therefore  the  expected  number  of  blocks  retrieved  by  a  signature  of  a  word  W 
of  text  (given  that  at  least  one  distinct  word  exists)  is  given  by 


k. 


D 

=!j  x;  - 


^ n( 

^.D  ^  ^  r  =  0 


N  —  — 
^  M 


N  -r 


-)) 


To  avoid  excessive  calculations  E{B)  can  be  approximated  by  expanding 
B[kl  ,N ,M)  around  the  expected  value  of  kl  [Papoulis].  Keeping  only  the  first 


term  of  the  expansion  we  obtain 


n-l 

E{B)  =  B{n,NM)  =  M{1  -  IK - ))  (2) 

r=0  ^ 

where  the  expected  number  of  words  n  that  are  retrieved  from  the  signature 
S{W)  of  an  existing  word  W  is  given  by 


(3) 


Since  a  document  may  span  more  than  one  block,  this  formula  assumes  that  all 


the  blocks  of  messages  that  contain  at  least  one  occurence  of  the  word  W  in  the 
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pattern  or  another  JV  'with  S{W)  -  S{W).  For  small  documents  this  assumption 
will  not  introduce  serious  errors. 

Given  a  text  file  of  N  non-trivial  words,  'wdth  D  distinct  words,  occupying  M 
blocks  of  secondary  storage,  the  expected  number  of  blocks  that  are  retrieved 
when  a  single  word  pattern  is  specified  can  be  estimated  using  (2)  and  (3).  Let 
B{Tt,N,M)  be  the  average  number  of  blocks  that  are  retrieved  when  a  signature 
of  a  single  word  is  specified.  Users  may  specify  patterns  that  are  regular 
expressions  of  signatures.  In  the  case  that  regular  expressions  are  restricted  to 
conjunctions,  disjunctions  (for  synonyms),  or  sequencing  (words  that  are  con¬ 
secutive  in  the  text  file)  the  average  number  of  blocks  retrieved  by  a  query  is 
given  by: 

1=2  i=2  i=2 

where 


r  is  the  frequency  of  single  word  patterns, 

is  the  frequency  of  patterns  specifying  conjunctions  of  i  words  of  text, 
is  the  frequency  of  patterns  specifying  disjunctions  of  i  words  of  text. 


is  the  frequency  of  patterns  specifying  sequences  of  i  words  of  text. 


n 


N_ 

C 


712  = 


N 


C 


i 


713  ~ 


-) 


i-1. 


c 


-)') 


The  first  summation  in  (4)  corresponds  to  conjunctive  patterns.  The  second 
summation  corresponds  to  sequencing  patterns.  Its  derivation  is  based  on  the 
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observation  that  given  a  sequence  of  signatures  the  next  signature  is  one  of  the 
existing  distinct  signatures  in  the  signature  file.  From  (3)  we  see  that  the 
expected  distinct  signatures  in  the  signature  file  is  given  by 


D,  =  C(l-(1-^)'’). 

n  C 

The  third  summation  corresponds  to  disjunctive  patterns.  Its  derivation  is 
based  on  the  fact  that  we  select  signatures  for  i  words  from  an  existing  set  of 


C(l— (l— distinct  signatures.  (This  also  can  be  modelled  as 


a  selection  with 


replacement  problem.)  The  above  formulae  assume  uniform  frequency  of  words 
and  independence  of  the  words  specified.  These  are  pessimistic  assumptions 
([Christodoulakis  a],  [Christodoulakis  c]). 


The  total  average  cost  is: 


C  =  CS*BS  -H  CT^BT 
Or 

C  =  CS*BS  +  CT*{rB{n^,N,M)  +  - -7^ -  (5) 

i=2  ^ 

+  +  V  dj 5 (713,7V, iV)) 

t=2  i=2 

.where  CS  is  the  cost  of  sequentially  scanning  a  block  of  the  signature  file,  CT  is 
the  cost  of  randomly  accessing  one  block  of  the  text  file,  BS  is  the  number  of 
blocks  that  are  used  for  storing  the  signature  file,  and  BT  is  the  average  number 
of  blocks  accessed  from  the  text  file  as  given  by  (4),  (3),  and  (2). 

One  way  to  reduce  the  cost  of  the  sequential  scan  of  the  signature  file  is  to 
reduce  the  size  BS  of  the  signature  file  by  removing  all  duplicate  word  signa¬ 
tures  from  the  signature  of  the  body  of  a  message.  Thus  the  signature  of  the 
body  of  a  message  will  be  composed  of  a  set  of  distinct  word  signatures.  In  this 
case  queries  involving  sequencing  of  words  can  be  satisfied  by  retrieving  the 
messages  containing  the  conjunction  of  the  given  words  and  discarding  the  non¬ 
qualifying  messages.  Since  the  number  of  messages  that  qualify  in  conjunctions 
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of  words  is  small  on  average,  this  technique  may  reduce  the  average  cost  C, 


CT 

In  (5)  - depends  on  the  de\'ices  where  the  signature  file  and  the  text  file 

CS 


are  stored.  It  also  depends  on  the  size  of  the  bufTer  and  the  block  prefetching 
and  replacement  algorithms  used.  Finally  it  depends  on  the  way  that  data 
blocks  of  the  signature  file  are  stored  on  the  secondary  device  (if  logical  order 
corresponds  to  physical  order).  All  these  factors  can  affect  considerably  the 
CT 


ratio 


CS 


Unfortunately  the  operating  system  support  for  these  functions  is  not 


satisfactory.  (For  example  LRU  is  the  worst  possible  replacement  algorithm  for 
scanning  the  signature  file.)  However,  the  use  of  large  blocking  factors  in  the 
signature  file  helps  because  this  results  in  natural  prefetching. 

The  size  of  word  signatures  should  be  chosen  such  that  (5)  is  minimized. 

Figure  (l)  shows  the  average  cost  BS  +  F*B{ni,N ,M)  as  function  of  the 


number  of  bits  used  for  the  signatures  of  words.  In  this  function  F  takes  into 


account  the  ratio 


CT 

CS 


of  costs  of  accessing  a  block  of  the  text  file  versus  the 


cost  of  accessing  a  block  of  the  signature  fi.le,  as  well  as  the  frequency  of  various 
types  of  patterns  in  queries.  (This  formula  is  accurate  only  when  a  small 
number  of  words  in  disjunctions  is  specified.)  The  optimum  choice  of  the  size  of 
signatures  depends  on  all  the  above  factors.  However,  as  the  figure  shows,  for  a 
large  range  of  values  of  the  parameter  F,  the  minimum  cost  is  achieved  for  sig¬ 
natures  neeir  15  or  16  bits.  This  is  also  true  for  larger  values  of  F  that  are  not 
shown  in  the  figure,  and  it  is  due  to  the  fact  that  for  this  signature  size  the  pro¬ 
bability  of  two  words  having  the  same  signature  becomes  small.  Additional 
experimentation  has  shown  that  the  choice  of  the  optimal  size  is  also  relatively 
insensitive  to  the  change  of  the  text  block  size.  For  8  bit  byte  machines,  the 
choice  of  16  bit  signatures  has  the  additional  important  advandage  that  only 
byte  comparisons  eire  used,  and  thus  the  cost  of  comparisons  per  bit  is  reduced. 
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Figure  1:  Average  cost  OCS+F^B  as  function  of  the  signature  size  of  words 
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Thus  in  such  an  environment  16  bit  signatures  seem  to  be  the  best  choice. 

In  the  above  we  have  analyzed  a  pure  document  retrieval  environment,  Le. 
messages  are  only  retrieved  on  the  basis  of  a  pattern  of  words  P .  Extensions  of 
this  analysis  to  include  the  case  where  values  for  some  attributes  are  also 
specified  in  the  retrieval  has  to  take  into  account  the  average  selectivities  of  the 
attributes  specified  ([Hammer  and  Chan  1976],  [Christodoulakis  a]). 

To  conclude  this  section  we  mention  that  this  file  organization  requires  less 
storage  than  word  indexing  methods,  it  makes  efficient  use  of  small  degress  of 
multiprogramming,  it  uses  just  one  pass  of  the  signature  file  in  order  to  retrieve 
pointers  to  documents  even  when  many  words  are  used  in  patterns  for  identify¬ 
ing  documents,  and  finally  it  avoids  merging  pointers.  Moreover  insertion  of  new 
documents  is  very  easy  (just  append  the  signature  of  the  new  document  in  the 
existing  signature  file),  while  if  indexes  are  used  the  index  has  to  be  updated  for 
every  word  of  the  new  document.  More  detailed  performance  comparisons  with 
a  word  indexing  method  can  be  done  by  using  (5). 

6.  Implementatioii 

We  are  implementing  a  message  filing  facility  as  discussed  in  this  paper 
based  on  UNIX.  The  access  method  based  on  signatures  has  been  implemented 
and  tested.  The  user  can  retrieve  documents  based  on  patterns  of: 

a)  A  single  word 

b)  A  sequence  of  words 

c)  All  the  words  of  a  given  set  of  words 

d)  One  or  more  words  of  a  given  set  of  words. 

Presently,  combinations  of  these  patterns  are  not  allowed. 

The  signature  file  is  created  after  the  elimination  of  a  set  of  100  common 
words.  In  this  implementation  signature  file  and  text  file  reside  on  the  same 
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device.  The  facility  will  be  enhanced  with  statistics  gathering  mechanisms  for 
studing  its  performance.  We  are  going  to  integrate  this  facility  into  a  message 
management  system. 


We  envision  the  facility  in  three  difierent  environments. 

In  the  first  environment  we  assume  a  workstation  running  UNIX,  e.g.,  ONYX, 
ii/-Iab,  LiSI  11/24,  witli  a  sniall  20  lib  disk.  Suppose  that  we  can  devote  up  to 
10Mb  of  the  disk  for  message  filing.  We  feel  that  we  should  be  able  to  store  of 
the  order  of  10,000  messages  of  less  then  a  page  length  on  the  average.  The 
messages  will  be  organized  in  approximately  20  different  logical  files  which  will 
hold  about  500  letters  each.  The  signature  of  such  logical  files  will  be  50  Xb 
which  can  be  searched  quite  fast  even  by  a  workstation  processor. 


In  the  second  environment  we  assume  a  message  file  server  operating  on  a 
high  power  microprocessor,  e.g.,  M68000  based  system  like  APOLLO  with  a  300 
Mb  disk.  This  facility  can  serve  approximately  20  users  connected  to  the  server 
via  a  local  network.  Each  user  can  store  on  the  average  10,000  messages  with 
similar  organization  and  response  as  in  the  first  environment.  The  total  capa¬ 
city  of  the  message  file  server  is  of  the  order  of  200,000  messages  which  can 
probably  be  adequate  for  a  department  or  groups  of  persons. 


In  the  third  environment  we  envision  a  high  power  microprocessor  with  a 
300  Mb  disk  storing  only  the  signatures.  In  such  an  environment  we  can  go  up  to 
2,000,000  messages.  The  messages  themselves  are  stored  on  microfilm  using  an 
addressable  reader  printer,  e.g.,  KODAK.  In  such  an  environment  the  user 
specifies  his  request  over  a  network  and  gets  the  associated  messages  using  a 
telefa_x  facility.  Such  an  environment  will  probably  be  well  suited  for  organiza¬ 
tional  archives  of  messages. 

Notice  that  in  all  three  cases  the  software  for  searching  for  the  messages  is 
the  same.  We  can  see,  therefore,  a  nice  combination  of  facilities  or  a  natural 
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transition  from  one  environment  to  the  other. 

We  should  also  mention  that  if  some  messages  are  not  in  machine  readable 
form  they  can  still  be  read  using  OCR  which  is  probably  centralized.  In  such  a 
case  messages  can  go  as  they  ceime,  on  say  microfilm,  while  their  signatures  are 
produced  and  retained  for  searching  purposes. 

Finally,  we  should  point  out  that  since  the  problem  of  document  retrieval  is 
reduced  to  membership  problem  of  regular  expression  the  whole  technique  can 
be  cast  into  VLSI  [Floyd  and  Ullman,  Foster  and  King]. 
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ABSTRACT 

In  an  office  information  system  environment,  -we  examine 
issues  related  to  the  design  of  a  message  file  server  ■which  allows 
for  retrieval  of  messages  based  on  contexf,.  Performance  con¬ 
siderations  in  this  environment  are  discussed,  and  the  analytic  for¬ 
mulas  for  the  optimal  choice  of  the  parameters  of  the  design  and 
the  reorganization  are  derived. 

1.  Introduction 

Consider  an  information  system  environment  and  the  messages  that  circu¬ 
late  within  it.  Messages  consist  of  a  header  and  a  body  [Kirstein,  ’80].  The 
header  contains  formatted  data  representing  the  most  important  characteris¬ 
tics  of  the  messages,  e.g.,  sender,  date,  destination,  etc.  The  body  is  text  con¬ 
sisting  of  series  of  words. 
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It  is  not  necessary  that  all  the  messages  be  of  the  same  type.  Each  type  of 
message,  howev'er,  is  represented  by  a  set  of  attributes  and  a  body.  Pure 
records,  forms,  documents,  letters,  memos,  and  renorts  can  all  be  seen  as  spe¬ 
cial  case  of  messages. 

Messages  in  office  environments  are  organized  in  a  hierarchical  structure, 
e.g..  folders  within  drawers  of  file  cabinets.  IVe  could  file  messages  in  an  elec¬ 
tronic  filing  environment  in  a  similar  way,  using  complex  hierarchical  direc¬ 
tories.  However,  there  are  serious  limitations  in  such  a  filing  facility  [Tsichritzis 
and  Christodoulakis,  ’83].  We  have  chosen  to  file  messages  in  general  files 
rather  than  complex  directories  e.g,  1981  general  correspondence,  project  X 
memos,  etc.  Within  each  logical  file  an  information  retrieval  capability  enables 
the  user  to  retrieve  quickly  the  desired  messages. 

The  file  organization  consists  of  a  two-level  hierarchy.  The  first  level 
{access  leueL)  provides  a  filtering  capability.  Its  purpose  is  to  restrict  the 
amount  of  search  required  on  the  second  level  (storage  level).  Messages 
retrieved  from  the  storage  level  form  a  superset  of  all  the  messages  qualifying  in 
the  query,  because  the  filtering  capability  of  the  access  level  is  not  exact.  Thus, 
messages  which  are  retrieved  from  the  storage  level  are  examined  for 
qualification  before  they  are  returned  to  the  user. 

The  two-level  file  organization  described  does  not  necessarily  imply  a  two- 
level  storage  hierarchy.  The  same  software  can  be  used  in  a  variety  of  environ¬ 
ments  [Tsichritzis  and  Christodoulakis.  ’83].  However,  the  small  size  of  the 
access  level  file  In  comparison  to  the  size  of  storage  required  when  a  word  index¬ 
ing  method  is  used  [Haskin,  ’Bl]  makes  the  approach  appropriate  for  storing 
large  organizational  archives  of  messages.  Consider  a  typical  office  information 
system  environment  with  workstations  running  UNIX,  interconnected  with  a  local 
network.  A  high  power  microprocessor,  e.g.  a  M6800  based  system,  is  used  as  a 
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back-end  message  file  server.  In  snch  an  environment,  the  entire  access  level 
file  can  be  stored  in  a  300  Mb  disk.  The  storage  file  can  be  stored  in  a  very  Large 
capacity  device  (for  example,  on  videodisk  or  microfilm  using  an  addressable 
reader  printer,  e.g.,  KODAX).  In  this  dedicated  environment  the  use  of  sequen¬ 
tial  scan  for  accessing  blacks  of  the  secondary  storage  presents  particular 
advantages  because  it  avoids  "seeks"  of  the  access  mechanism. 

In  this  paper  we  examine  design  considerations  for  the  message  file 
server.Various  aspects  of  the  design  are  discussed  and  analytic  formulas  for 
design  and  reorganization  are  derived. 

2.  Access  Pile  Design 

In  the  storage  file,  messages  are  stored  sequentially.  In  the  access  file  sig- 
natv.res  of  messages  are  stored  sequentially.  Consider  a  message  m 
represented  by  (fio.ai,  •  •  •  .a^.b)  where  ao.a^,  •  •  •  are  the  attribute  values  of 

the  attributes  of  the  header  eind  b  is  the  body  of  the  message  (of  type  text).  The 
signature  of  m,  S{m)  is  represented  by  ,5(ao),5(a,j)... .,5(ti^),5'(b  )),  where  t 
indicates  the  type,  S  {aQ),S{ai),  •  ■  ■  ,5'(a^)  are  the  signatures  of  aQ,ai,  •  •  •  .a^, 
and  S{b)  is  the  signature  of  the  body. 

A  query  specifies  the  type  of  the  message,  and  optionally  specifies  the 
values  for  some  of  the  attributes  as  well  as  some  pattern  P  of  words.  The  pat¬ 
tern  P  of  words  can  be  a  single  word,  a  part  of  a  word,  or  a  complicated  regular 
exp^'ession  of  words  and  parts  of  words.  Messages  qualifying  In  the  query  are 
those  that  (l)  are  of  the  same  type,  (2)  satisfy  the  conjuction  of  the  attribute 
values  specified,  and  (3)  the  pattern  P  appears  in  their  body. 

To  find  the  set  of  qualifying  messages,  the  access  file  is  searched  sequen¬ 
tially.  The  type  of  a  message  is  examined  first.  If  the  type  of  the  message  does 
not  agree,  the  remainder  of  the  signature  of  the  message  is  skipped.  Next,  the 
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signature  of  the  attributes  specified  in  the  query  are  examined  for  qualificatiorL 
Finally,  if  the  message  is  of  the  same  type  as  the  one  specified  in  the  query  and 
its  attributes  satisfy  the  conjuction  of  the  conditions  specified  in  the  query,  the 
signature  of  the  body  of  the  message  is  searched  to  determine  whether  the  mes¬ 
sage  itself  should  be  retrieved  from  the  storage  file. 

Signatures  in  our  context  provide  a  partial  match  retrieval  capability. 
Their  purpose  is  to  provide  information  about  the  contents  of  messages  in  much 
less  space  than  the  space  that  the  messages  need.  Signatures  of  attributes 
values  should  be  attribute  domain  dependent.  Intuitively,  attributes  with  a 
small  number  of  attribute  values  should  require  less  space  for  signatures  than 
attributes  with  a  large  number  of  attribute  values.  Moreover,  for  numeric  attri¬ 
butes,  the  signature  construction  method  should  be  such  that  inequality  queries 
can  be  (partially)  answered.  We  shall  not  examine  attribute  signatures  in  more 
detail  here.  For  the  purpose  of  this  paper,  signatures  of  attribute  values  will  be 
considered  to  be  exact  representations  of  the  attribute  values  themselves  i.e. 
5'(ao)  =  ao,5'(ai)  =  ai,  •  •  •  ,5'(a„)=a^.  In  the  remainder  of  this  section,  we  shall 
describe  the  method  used  for  the  construction  and  the  search  of  the  signatures 
of  the  body,  as  well  as  for  the  choice  of  the  parameters  in  our  design.  In  many 
cases,  the  storage  requirements  for  the  signatures  of  bodies  of  messages  will  be 
considerably  more  than  the  storage  requirements  for  the  signatures  of  the  attri¬ 
bute  values. 

Message  size  may  vary  considerably  from  message  to  message.  Thus  the 
body  of  a  message  may  span  from  one  to  several  blocks  of  secondary  storage  in 
the  storage  file.  Let  61,62,  •  •  ■  ,6„  be  a  partition  of  the  body  6  of  a  message  m, 
such  that  61,62,  ■  ■  •  ,6„  are  stored  in  separate  blocks  of  the  storage  device, 
where  the  storage  level  file  resides.  The  signature  of  the  body  S{b)  is  also  parti¬ 
tioned  into  block-body-signatures  5'{6  i),5'(62),  •  •  •  ,S(bn)  according  to  the  par- 
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titions  of  the  body  b. 

A  block-body  signature  5'(6i)  of  a  block  b^  of  the  body  of  a  message  is  a  set 
of  F  bits  formed  as  follows.  Originally  the  F  bits  are  set  to  zero.  Punctuation 
marks  arid  formatting  symbols  do  not  appear  in  the  signature.  Similarly,  the 
common  words  of  do  not  appear  in  the  signature  S(b^}.  (The  system  main¬ 
tains  a  dictionary  of  common  words.)  Each  non-common  word  of  6^  is 
represented  in  S(bi)  by  a  set  of  m  bits  which  are  set  to  ”1".  In  order  to  deter¬ 
mine  which  bits  should  be  set,  a  hashing  technique  is  used.  We  shall  describe 
the  choice  of  the  hashing  function  later.  Note  that  these  m  bits  may  not  be  dis¬ 
tinct.  F  and  m  are  design  parameters. 

To  determine  if  a  given  word  zu  exists  in  a  block  6^  of  the  body  of  a  message, 
the  system  applies  the  same  transformation.  If  all  the  indicated  bits  are  "I"  in 
S(bi),  the  word  is  assumed  to  appear  in  b^.  It  is  clear  from  the  above  descrip¬ 
tion  that  the  set  of  blocks  that  appear  to  contain  the  word  tu  is  a  superset  of 
the  set  of  blocks  that  actually  contain  w.  More  complicated  patterns  of  words 
like  conjunctions,  disjunctions  and  sequencing  can  be  examiined  in  the  obvious 
manner.  Sequencing  of  two  words  for  example  can  be  handled  by  examining 
whether  the  bits  which  are  set  by  both  words  are  set  in  the  signature  of  the 
block.  Thus,  blocks  of  messages  satisfying  the  conjunction  of  the  two  words  are 
retrieved  and  checked  before  they  are  passed  to  the  user.  If  a  conjunction  of 
words  specified  in  a  pattern  is  satisfied  by  words  existing  in  more  than  one 
blocks  of  the  message,  all  the  relevant  blocks  may  have  to  be  shown  to  the  user. 
(See,  however,  section  3  ).  This  method  of  accessing  messages  based  on  the  con¬ 
tents  of  the  body  of  the  message  resembles  various  superimposed  coding  tech¬ 
niques  which  have  appeared  in  the  literature  [Orosz  and  Tackacs,  '56,  Taube  et 
al.,  '57.  Stiassny,  '60,  Goldgerg  et  al.,  '61.  Roberts,  '79.  Pflatz  etaL,  '80] 

This  method  of  construction  of  block  signatures  provides  for  automatic 
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elimination  of  duplicate  words  within  a  block  of  text.  In  comparison,  a  method 
based  on  word  signatures  would  have  to  eliminate  duplicates  by  sorting.  More¬ 
over,  less  space  is  required  for  a  block  signature  method  than  for  a  word  signa¬ 
ture  one.  Alternatively,  message  signatures  could  be  provided  instead  of  block 
signatures.  However,  the  partitioning  of  messages  into  fixed  size  units  (blocks) 
allows  for  a  good  choice  of  the  design  parameters  F  and  m.  If  these  parameters 
were  selected  for  an  "average"  message  size,  performance  would  deteriorate  due 
to  the  high  variance  of  message  sizes.  Finally,  another  reason  for  the  partition¬ 
ing  is  that  the  filtering  capability  of  the  access  file  is  not  exact.  Thus,  in  our 
design,  the  signature  of  the  body  may  indicate  that  a  pattern  P  of  words  exists 
In  the  signature  of  the  body  of  a  message,  when  actually  it  does  not.  In  this 
case,  the  cost  incurred  is  the  cost  of  retrieving  a  subset  of  the  blocks  of  the 
message  instead  of  the  whole  message. 

Next,  we  shall  describe  the  estimation  of  the  design  parameters  F  and  m. 
Intuitively,  a  small  F  will  result  in  a  large  superset  of  blocks  appearing  to 
have  a  given  word  w,  while  a  large  F  will  result  in  an  increase  in  the  cost  of 
sequentially  scanning  the  access  file. 


2.1  Optimization. 

In  this  section  we  ignore  the  existence  of  attributes  and  concentrate  on  the 
retrieval  of  the  messages  based  on  context.  We  shall  examine  the  effect  of 
retrieval  based  on  attribute  values  later  on.  Our  purpose  is  to  estimate  the 
parameters  F  and  m,  such  that  the  cost  is  minimized.  (Table  D  gives  the 
definitions  for  the  symbols  that  we  are  going  to  use  in  the  this  analysis.)  The 
total  system  cost  for  answering  a  query  that  is  expressed  as  a  pattern  consisting 
of  a  single  word  is 


F 


(BF) 


Btot  ( 


{C5)  +  /’s(CT)) 


(1) 
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Symbol  Definition. 


Fi’.  Number  of  bits  of  the  signature  of  a  block 


771 :  Number  of  bits  that  a  'W'ord  sets  to  "1"  (not 


necessarilly  distinct.) 


Expected  number  of  distinct 


non-common  words  in  a  block. 


M:  Expected  number  of  ”1”  ’s  in  a  block  signature 


containing  Di,i  distinct  non-common  words. 


Probability  that  a  block  signature  qualifies 


in  a  single  word  query  ("dropping  fraction"). 


CS:  Cost  of  accessing  a  single  block  of  the  access  file. 


CT'.  Cost  of  accessing  a  single  block  of  the  storage  file. 


Qot  •  total  cost  of  a  single  word  query. 


Total  number  of  blocks  in  the  text  file. 


BF:  Number  of  bits  in  a  block  of  the  access  file. 


Table  D. 


We  mention  here  that  CS  can  be  considerably  smaller  than  CT,  even  if  both 
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files  are  stored  in  the  same  device.  The  reason  is  that  CS  is  greatly  reduced  by 
the  use  of  large  logiccd  blocking  factors,  which  result  in  natural  prefetching. 
Moreover,  since  only  a  small  number  of  blocks  are  retrieved  from  the  storage 
file,  the  rotational  delay  per  block  retrieved  is  larger  on  average. 


We  shall  calculate  F4  as  a  function  of  F  and  m.  To  do  this  we  need  to  know 
the  probability  distribution  of  ”1”  's  in  the  F  bits  of  the  signature,  given  that  Diji 
distinct  non-common  words  exist  in  a  block  of  the  body  of  a  message. 

Then  F^  is  given  by: 


F  r/' 


fi'=0 


(2) 


where  Pr{M' is  the  probability  that  M'  are  set  to  "1"  in  a  signature  S{b^)  of 

a  block  bi  when  D^i  distinct  non-common  words  exist  in  the  block,  and  (  -pr')”'  is 

F 

the  probability  that  a  signature  of  F  bits  with  M'  "1”  's  qualifies  in  a  single-word 


query. 


Since  we  have  assumed  that  the  m  bits  set  to  ’"I'’  by  a  word  may  not  be 
distinct,  the  problem  of  estimating  Pr{M\D^i)  can  be  modelled  as  a  problem  of 
selecting  with  replacement  Tn*Dbi  bits  from  a  set  of  F  bits.  This  problem  can  be 
modelled  using  Markov  chains  [Klemrock  ’75].  Each  state  is  described  with  the 
number  of  bits  of  the  signature,  which  have  been  set  to  "1".  In  figure  1,  the 
states  of  the  Markov  chain  and  the  associated  transition  probabilities  are  shown. 
Let  T  be  the  state  transition  matrix: 


T  = 


0 

0 


1  0 
1_  F-1 
F  F 


0 

0 


0  0  0 


F 


The  probability  distribution  of  the  number  M'  of  ”1"  ’s  after  m*Diyi  have 
been  selected  is  expressed  by  the  state  probability  vector: 
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Pm-B„  =  {P(0.m-Dii).  . P{F.Tn.-D^^)) 

P.T.-0,,  =  (3) 

where  P{k,l)  is  the  probability  that  k  bits  are  set  to  ”1”,  after  I  random  bit 

selections  and  Pq  is  the  vector  (1,0,  •  •  •  ,0)  (with  Fkl  elements). 

The  estimation  of  using  (2)  and  (3)  is  exact,  but  it  is  inconvenient  for 
optimization.  A  simplified  formula  can  be  obtained  by  expanding  (2)  around  the 
mean  M  of  the  distribution  of  M'  [Papoulis  ’65]  and  keeping  the  first  term.  M  is 
the  expected  number  of  "1"  ’s  after  the  selection  with  replacement  of  Tn*D(ji 
bits.  Therefore 


and 


//  =  F(1-(1- 


F' 


F<i  =  (f-)"  = 


m 


Substituting  in  (l),  we  obtain: 


Figure  3  shows  the  cost  function  Ctot  ^is  a  function  of  F  for  various  values  of 

the  ratio  Qosed  form  formulas  for  approximating  the  values  of  F  and  m 

(CS) 

which  minimize  the  cost  are  derived  in  Appendix  1.  Table  1  compares  the 
optimal  values  with  the  values  given  by  the  formulas  in  Appendix  1.  In  all  cases 
the  approximation  of  the  optimal  values  is  very  good. 


3. Choice  of  Haishmg  P\mction. 

We  have  not  discussed  so  far  the  details  of  the  implementation  of  the  hash¬ 
ing  function.  In  fact,  a  variety  of  hashing  methods  can  be  used  for  determining 
the  m  bits  which  are  set  by  the  signature  of  a  word.  In  this  section  we  present  a 
method  for  determining  the  bits  within  the  signature  of  a  block  that  are  set  by  a 
given  word. 
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We  want  to  allow  the  user  the  flexibility  of  specifying  parts  of  words  inside  a 
pattern  of  words  P.  For  the  purpose  of  this  paper,  we  restrict  the  queries  to 
peirts  of  words,  which  steirt  from  the  first  letter  of  the  word.  In  our  environment, 
we  expect  that  the  users'  requests  will  be  as  such.  An  arbitreiry  hashing  function 
does  not  allow  for  parts  of  words  to  be  specified  in  queries.  We  need  to  use  a 
form  of  "deterministic”  hashing,  which  will  encode  much  information  about  the 
sequence  of  letters  inside  words.  We  have  chosen  triplets  of  consecutive  letters 
to  encode  the  sequencing  information.  Triplets  of  consecutive  letters  are  known 
to  contsdn  a  great  deal  of  information  about  English  words,  much  more  than  sin¬ 
gle  letters  [Abramson,  *63].  It  follows  that  by  encoding  information  about  con¬ 
secutive  triplets  inside  a  word,  we  encode  much  information  about  the  word.  In 
addition,  the  number  of  possible  triplets  is  manageable.  Thus  we  do  not  have  a 
search  problem  (as  it  would  be  the  case  if  we  were  encoding  longer  sequences  of 
letters). 

Consider  am  arbitrary  word  of  text.  The  word  is  separated  into  triplets  as 
follows:  The  first  triplet  is  formed  from  the  three  first  letters  of  the  word.  The 
second  from  the  second,  third  and  fourth  letters  of  the  word  and  so  on.  If  the 
number  of  triplets  exceeds  the  number  m,  of  bits  allowed  per  word,  the  remain¬ 
ing  letters  are  omitted.  For  small  words  where  the  number  of  triplets  of 
letters  I  as  defined  above  is  less  than  m,  the  remaining  m—l  bits  have  to  be 
specified  using  another  method.  One  way  to  do  it  is  to  use  a  random  number 
generator,  initialized  by  a  suitable  bit  representation  of  the  word. 

According  to  the  above,  a  given  triplet  of  letters  maps  to  a  given  bit 
number  (within  the  F  bits  composing  a  block  signature).  It  is  important  that 
the  triplets  of  a  large  text  are  uniformly  mapped  among  all  F  bits.  The  relative 
frequencies  of  triplets  of  words  in  a  l6Lrge  English  text  are  known  [Abramson,  ’63] 
Our  problem  therefore  is  to  map  these  relative  frequencies  on  F  bits  in  a  uni- 
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form  way.  We  use  the  entropy  of  the  resulting  frequency  distribution  as  a  meas¬ 
ure  of  the  uniformity  of  the  distribution.  An  optimal  mapping  is  defined  to  be 
one  which  maximizes  the  entropy  of  the  resulting  frequency  distribution. 

Let  =  be  the  sum  of  the  relative  frequencies  of  the  triplets  which 

i 

map  to  the  _7-th  bit  of  the  block  signature.  Yfe  want  to  choose  the  mapping 
which  maximizes  the  cost  function: 

F 

y=i 

This  mapping  problem  resembles  a  class  of  difTLcult  problems,  which  can  be 
described  as  follows:  Allocate  k  objects  with  known  frequencies  into  F  buckets 
(with  infinite  capacity  each,  in  our  case)  such  that  a  cost  function  is  optimized. 
Some  of  these  problems  are  already  known  to  be  in  the  class  of  NP-complete 
problems.  In  our  case  however  the  cost  function  has  a  specific  property,  which 
makes  the  developement  of  heuristics  for  the  allocation  easier.  Specifically,  our 
cost  function  is  Schur  concave  [Wong  and  Yue,  ’73].  Schur  concave  functions 
have  certadn  monotonicity  properties,  which  find  important  application  in  Com¬ 
puter  Science  ([Wong  and  Yue,  ’73],  [Christodoulakis  ’82].  This  property  of 
entropy  suggests  the  following  heuristic  algorithm  for  mapping  triplets  into  bits: 

Algorithm  A 

1.  Order  triplets  according  to  frequency, 

2.  Start  from  the  most  frequent  triplet. 

Map  next  triplet  to  the  bit  with  the  smallest  frequency  so  far. 

3.  Repeat  step  2  until  there  are  no  more  triplets. 

Algorithm  A  has  the  property  that  the  choice  at  each  step  of  the  iteration  is 
optimal  in  the  sense  that  any  other  mapping  at  this  step  would  produce  higher 
entropy  (due  to  the  Schur  concave  property  of  the  entropy  function).  Although 
Algorithm  A  chooses  an  optimal  mapping  at  each  step,  it  does  not  quarantee  a 
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global  optimum.  Given  that  a  suboptimai  allocation  may  result,  an  important 
question  is  how  far  the  cost  of  the  solution  of  Algorithm  A  is  from  the  cost  of  an 
optimal  solution.  More  research  is  required  in  this  area. 

4.  Access  rae  Reorganization. 

There  are  several  reasons  for  reorganizing  the  access  file.  An  office  infor¬ 
mation  system  environment  is  a  changing  one.  The  interests  of  the  company 
change  as  new  projects  are  started.  It  has  been  observed  that  the  access  fre¬ 
quency  of  a  given  message  decreases  exponentially  with  time  [Gravina,  *78].  In  a 
message  environment  the  access  frequency  should  also  be  a  function  of  the  mes¬ 
sage  type. 

Consider  a  message  m  which  spans  several  blocks.  The  access  frequency  of 
all  the  blocks  is  not  the  same.  Most  of  the  important  words  of  a  document  will 
appear  in  the  first  few  pages  of  the  document.  Usually,  the  user  is  able  to  say  if 
a  message  qualifies  or  not  by  just  looking  in  the  first  page  of  a  message.  If  the 
message  qualifies,  the  user  asks  to  see  it  €l11,  otherwise  he  wants  to  skip  the  mes¬ 
sage.  In  both  cases  the  signature  of  the  blocks  skipped  is  not  used.  Occasion¬ 
ally,  the  user  is  not  certain  and  he  accepts  to  see  more  message  contents  if  they 
contain  the  pattern  of  words  that  he  has  specified.  Clearly,  information  about 
the  frequency  of  the  use  of  the  block  signatures  does  not  exist  at  message  inser¬ 
tion  time. 

Finally,  consider  messages  which  are  mainly  retrieved  based  on  the  attri¬ 
bute  values  of  their  attributes.  For  these  messages,  the  signature  of  the  body 
may  not  be  as  useful  in  retrieval,  if  the  attributes  are  very  selective.  Again,  this 
information  may  not  be  known  at  message  insertion  time.  Thus,  access  file  reor¬ 
ganization  according  to  the  frequency  of  use  of  the  signature  of  the  body  of  mes¬ 
sages  could  improve  performance. 
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IfVe  want,  however,  the  reorganization  to  be  inexpensive  and  to  result  in  a 
uniform  way  of  examining  signatures.  In  the  following  ,  we  assume  that  the  fre¬ 
quency  of  use  of  a  signature  S{bi)  for  message  retrieval  can  only  decrease  from 
one  reorganization  interval  to  another.  This  is  compatible  with  the  fact  that  the 
frequency  of  retrieval  of  a  given  document  is  a  decreasing  function  of  its  age. 

Let  fi  be  the  relative  frequency  of  using  the  signature  S{b^)  for  retrieving  a 
block  6^  of  the  body  of  a  message  from  the  storage  file.  In  the  previous  section 
fi  was  taken  to  be  1  (i.e.  blocks  of  messages  were  always  retrieved  by  examining 
■whether  the  signature  of  the  body  contains  the  pattern  of  words  P).  However,  as 
we  explained  above,  a  signature  5(6^)  may  be  skipped.  Thus  may  be  less  them 
1.  We  assume  here  that  we  update  the  relative  frequency  fy  in  every  user's 
query  (or  one  out  of  every  k  queries).  The  frequency  will  be  used  at  reorgani¬ 
zation  points  to  determine  the  new  size  of  the  signature  5(bi).  Very  small  fy 
should  obviously  imply  small  S (by)  or  even  no  signature  at  all. 

Let  Fy  be  the  size  of  the  signature  of  a  block  by  for  fi  =  l.  At  reorganization 
points,  the  new  signature  S’iby)  is  derived  from  S{bi)  by  simply  eliminating  the 
last  71  bits  of  S{bi).  The  optim-um  number  of  bits  n  is  a  design  parameter  and  it 
is  a  function  of  the  frequency  fy.  Since  it  has  been  observed  that  fi  only 
decreases  as  a  function  of  time,  the  signature  S"(bi)  at  a  new  reorganization 
point  can  be  derived  from  ^7  sliminating  the  appropriate  number  of  bits 

at  the  end  of  S'(bi).  Thus,  the  storage  file  is  not  touched  at  reorganization 
points.  It  is  easy  to  see  that  the  procedure  which  examines  whether  a  pattern  P 
of  words  is  contained  in  S'{bi)  is  the  same  as  before.  However,  since  the  last  n 
bits  of  S{bi)  have  actually  been  removed,  the  cost  of  the  sequential  scan  of  the 
access  level  file  has  decreased. 

The  reorganization  described  above  has  certain  advantages:  It  is  done  in 
one  file  pass  (sequential  scan)  of  the  access  file.  Since  the  access  file  is  smaller 
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in  comparison  to  the  storage  file,  the  cost  of  the  reorganjj?:ation  is  very  small. 
This  is  in  contrast  to  what  happens  in  other  data  base  environments,  where  reor¬ 
ganization  is  usually  expensive.  The  purpose  of  the  reorganization  as  described 
above  is  not  to  eliminate  overfiow  pointers  or  other  factors  which  deteriorate 
the  performance  in  typical  data  base  environments,  but  to  adapt  the  access  file 
to  the  changing  interests  of  users.  In  this  respect,  we  can  see  it  as  an  ’'adap¬ 
tion”  of  the  access  file  to  users’  queries.  This  is  not  easily  achievable  in  tradi¬ 
tional  acces  mechanisms  (like  B-trees).  Another  important  property  of  this 
method  is  that  each  word  ^£;  of  a  given  search  pattern  P  need  only  be  hashed 
once .  Based  on  the  result  of  this  hashing,  all  the  (variable  size)  signatures  S{b^) 
can  be  examined.  In  the  following  we  shall  derive  formulas  for  the  estimation  of 
the  parameters  of  the  reorganization. 

The  total  system  cost  which  corresponds  to  a  given  block  6^  is  given  by: 

=  (5) 

In  this  formula,  F— n  indicates  the  decrease  in  the  cost  of  scanning  the  access 
level  file,  while  F'^  indicates  the  increased  probability  of  retrieving  a  block  of 
the  storage  level  file. 

As  before,  the  problem  can  be  modelled  using  Markov  chains,  as  shown  in 
Figure  2.  The  state  transition  matrix  T'  is  given  by: 


T  = 


ZL 

F 

0 

0 


F  —n 
F 

n  + 1 
F 


0 


0 

0 

£ 

F 


The  probability  distribution  of  "I”  ’s  after  Tn*Dbi  bits  have  been  selected  is 
given  by: 


m*Dt 


=  T'"'°"Fo 


(6) 


129 


where  P'q  is  the  vector  (1,  0,  ....  0),  ■with  F-n-hl  elements.  A  simplified  formula 
for  the  expected  number  of  "I"  's  is  given  by: 


M'  =  M +Ti 


M 


F 


O) 


amd  the  dropping  fraction  F'^  is 


M' 


F 

V  * 


M  -t-n 


1- 


M 


F 


m 


(8) 


Substituting  (8)  into  (5),  we  find: 


M  +n 


F 


m 


In  order  to  find  the  optimal  size  of  the  signature  S'ib^)  we  solve  the  equation 


dn 


'tot 


=0 


(9) 


The  optimal  number  n  of  bits  to  be  disregarded  is  calculated  in  Appendix  2.  The 
resulting  formula  is: 


rF-M  r. 


(10) 


where: 


r  = 


aF^ 


m-l 


(11) 


[fmiF-M] 

Figure  4  shows  the  access  cost  versus  the  number  n  of  bits  to  be  disre¬ 
garded  for  various  values  of  f.  If  the  value  of  /  is  large  ,  then  we  have  to  keep  a 
large  number  of  bits  in  the  block  signature  (i.e.  the  optimum  n  is  small).  As  / 
decreases,  we  need  less  and  less  bits  or  we  need  no  signature  at  ail  (optimum 
n=F). 


Tables  2-6  show  the  calculated  optimal  n  for  various  values  of  the  ratio 


CT 

CS' 


They  also  show  the  relative  saving  in  the  access  cost  if  reorganization  is  used. 


130 


5.  Sunmiary- 

We  have  presented  a  message  file  server  facility  in  an  ofTics  information  sys¬ 
tem  environment.  Type  and  size  of  message  may  be  variable.  Messages  may  be 
composed  by  a  set  of  fixed  attributes  and  a  few  Imes  of  text,  or  they  may  be 
multipage  letters,  documents  or  forms.  The  file  organization  forms  a  two  level 
hierarchy.  The  first  level  (access  level)  is  an  abstraction  of  the  second  level 
(storage  level)  where  actual  messages  are  stored.  Design  and  reorganization 
issues  for  this  organization  have  been  presented.  Closed  form  formulas  for 
optimal  design  and  reorganization  have  been  derived. 

The  message  filing  facility  described  here  is  being  part  of  an  integrated 
office  information  system,  which  is  now  being  implemented  at  the  University  of 
Toronto  [Tsichritzis,  ’82]. 

Appendix  1. 

In  this  Appendix  we  choose  a  set  of  values  for  F  and  m,  such  that  the  cost 
Qo<  given  by  (4)  is  minimized.  Since  Mi^i  and  CT  are  constants,  we  can  minimize 
the  function: 


_  f  m _ N  _ 

“  {BF){CT) 

=  aF+F^ 

(Al) 

_  (cs) 

“  {CT){BF) 

(A2) 

(A3) 

(A4) 

m*D^l 

«F(l-e  ^  ) 

(A5) 

Differentiating,  we  obtain: 
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Equation  (A?)  gives: 


d 


dm 

a 

dF 


-g=0 

7=0 


(A6) 

(A7) 


dm 


■Fi=0 


or  [Stiassnj'’  ’60]: 


m 


-  ^ 


D. 


bL 


■where  log  is  the  natural  logarithm.  Eq  (A8)  is  equivalent  to: 

m*D^l 


F  - 


or,  due  to  (A5): 


^=f 


Equation  (A7)  is  equi’valent  to: 
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Tn*D„i 
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F  J 
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F 
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—e  ^  m*Di 
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F^ 
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a  =772. 
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F 
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^  m*Di 


bl 


F‘ 


(AS) 


(A9) 


(AlO) 


(All) 


Substituting  M  from  (AlO)  and  F  from  (A8),  "we  have,  after  some  algebraic  mani¬ 
pulations: 


or 


2m^(l22Dl 

aDi,i 
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(All) 


Solving  for  m,  -we  obtain; 


(A13) 


From  (A8)  "we  can  solve  for  F  : 


Equations  (A  12)  and  (A13)  give  the  optimum  values  for  our  design  parameters. 
Appendix  2. 

In  this  Appentix,  we  shall  find  the  value  of  n  that  minimizes  the  cost  func¬ 
tion  (5).  The  values  of  F,  m  and  f  are  given.  We  have  to  find  the  solution  of  the 
equation: 


on 

where  C  is  the  access  cost  per  block  after  the  reorganization  (Le.  after  n  bits 


from  the  block-signature  are  eliminated).  Obviously,  we  have: 


where  F'^  is  the  dropping  fraction  after  the  reorganization.  Letting: 


a:,. -ACS) 
{CT}(BF) 


equation  (A14)  gives; 


-a+f-l—FU=0 


(A15) 


The  dropping  fraction  F'^  after  the  reorganization  is  given  by: 


(A16) 


where 


> 


F-M 


n 


F 


(A17) 
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The  last  t’W'o  formulas  have  the  foUowing  justification:  Disregarding  the  last 


n 


bits  is  equivalent  to  setting  them  all  to  "1".  But  there  were  on  the  average 


rtM 

F 


1  s  already.  So,  the  signature  after  the  reorganization  "seems'’  to  have  IF  "1" 

were  turned  from  "0"  ’s  to 


•  —  it/ 

’s,  where  I^i'  is  equal  to  M  plus  the  - bits  that 

F 


'1"  ’s- 


Carrying  out  the  differentiations  on  eq.  (A15),  we  obtain: 


or 


Defining 


we  have; 


or 


/T7X 


M' 


F 


1  F-M 

F  F 


ML 

F 


clF‘^ 


fm.{F-M) 


m  -1 


r  = 


M' 


M  +n 


r  = 


F-M 

F 


F 


(A18) 


(A  19) 


(A20) 


 tF-M 


(Aai) 


The  eqs.  (A19),  (A20),  A(2l)  give  the  value  of  n  which  minimizes  the  access  cost. 
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FIGURE  1 

Meirkov  chain  for  the  bit  selection. 
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FIGURE  2 

Markov  chain  for  bit  selection 
after  the  removal  of  n  bits. 
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FIGURE  3 

Total  cost  per  block  as 
L  function  of  F. 
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FIGURE  4 

Cost  after  reorganization 
vs.  n. 


138 


CT/CS=31.  F=613.  m=ll 


freq.  f  } 

opt.  n 

sav.% 

.8 

0 

0 

.4 

39 

1.5 

O 
•  u 

59 

3.3 

.1 

141 

12.5 

.08 

158 

14.8 

.04 

216 

22.9 

.03 

241 

26.5 

.01 

nAA 

41.5 

.008 

366 

44.8 

.004 

439 

55.5 

.003 

470 

60.2 

.001 

600 

79.4 

.0008 

613 

83.4 

CT/CS=41.  F=634.  m=ll 


freq.  f 

opt.  n 

sav.% 

.8 

0 

0 

.4 

oo 

1.2 

.3 

56 

2.7 

.1 

136 

11.5 

b 

CD 

153 

13.8 

.04 

210 

21.4 

.03 

235 

24.3 

.01 

336 

39.1 

.008 

358 

42.2 

.004 

429 

52.4 

.003 

460 

56.9 

.001 

587 

75.1 

.0008 

615 

79.1 

.0004 

634 

99.4 

Table  5. 


Table  6. 


CT/CS 

F(estLm.) 

F(actual) 

m(estim.) 

rn(actual) 

1 

324 

327 

6 

6 

11 

523 

524  ' 

9 

9 

21 

577 

578 

10 

10 

31 

610 

613 

11 

11 

41 

633 

634 

11 

11 

Table  1. 


CT/CS=1.  F=327.  m=6 


freq.  f 

opt.  n 

sav.% 

.8 

0 

0.0 

.4 

45 

3.0 

.3 

69 

6.4 

.1 

173 

27.7 

.08 

197 

33.4 

.04 

279 

53.4 

.03 

316 

62.8 

.01 

327 

37.5 

Table  2. 


CT/CS=11.  F=524,  m=9 


freq.  f 

opt.  n 

sav.% 

.8 

0 

0 

.4 

35 

1.5 

.3 

57 

3.3 

.1 

141 

14.3 

.08 

160 

17.2 

.04 

221 

27.0 

.03 

243 

31.5 

.01 

361 

50.4 

.008 

386 

54.6 

.004 

468 

68.4 

.003 

504 

74.5 

.001 

524 

91.4 

CT/CS=21.  F=578.  m=10 


freq.  f 

opt.  n 

sav.% 

,8 

0 

0 

.4 

36 

1.3 

.3 

56 

3.0 

.1 

iod 

12.3 

.08 

156 

15.3 

.04 

215 

23.9 

b 

CO 

241 

27.8 

.01 

347 

44.1 

.008 

370 

47.7 

.004 

446 

59.5 

.003 

479 

64.7 

.001 

578 

85.1 

Table  4. 


Table  3. 
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